KVpop：基于预测性在线剪枝的键值缓存压缩

摘要

键值（KV）缓存增长是自回归解码的主要瓶颈，其内存和带宽随上下文长度线性增长。现有的KV驱逐方法通常依赖静态启发式规则或代理分数，这些方法难以有效追踪未来词元的效用，且随着相关性变化易导致驱逐决策不稳定。为解决此问题，我们提出KVpop——通过直接监督保留/丢弃决策来学习固定预算的KV驱逐策略。评分模型针对一种新颖的未来注意力目标进行训练，该目标可在无需实例化密集注意力图的情况下高效计算。我们进一步引入了一种基于延迟记忆的评分器，这在学习型驱逐方法中独具特色——它通过将评分延迟固定步数，以利用近期未来上下文信息。在AIME和HMMT数学推理任务中，KVpop在Qwen3-4B模型上以75%的KV缓存压缩比保留了全注意力性能的98%，在88%压缩比下保留了97%，持续优于现有驱逐基线方法。Qwen3-8B模型展现出更优的结果，达到了接近完整教师模型的性能。这些结果表明，利用未来注意力信号监督驱逐策略，可在降低内存成本的同时保持生成质量。

English

Key-value (KV) cache growth is a major bottleneck in autoregressive decoding, as memory and bandwidth scale linearly with context length. Existing KV eviction methods often rely on static heuristics or proxy scores, which poorly track future token utility and cause brittle eviction as relevance shifts. To address this, we introduce KVpop, which learns a fixed-budget KV eviction policy by directly supervising the keep-or-drop decision. The scorer is trained against a novel future-attention target, computed efficiently without materializing dense attention maps. We further introduce a delayed memory-based scorer that, uniquely among learned eviction methods, defers scoring for a fixed number of steps to exploit near-future context. On AIME and HMMT mathematical reasoning, KVpop retains 98% of full-attention performance on Qwen3-4B at 75% KV cache compression and 97% at 88% compression, consistently outperforming established eviction baselines. Qwen3-8B shows even stronger results, reaching near-full teacher performance. These results show that supervising eviction with future-attention signals cuts memory costs while maintaining quality.