BeaconKV：由 Beacon 查詢引導的鍵值快取壓縮，用於高效的大型推理模型推論

摘要

大型推理模型（LRMs）透過擴展式思維鏈（CoT）生成，達成卓越的解題表現，但所產生的鍵值（KV）快取會隨序列長度呈線性成長，造成嚴重的記憶體瓶頸，在長推理軌跡下往往超出 GPU 容量。現有的 KV 快取壓縮方法依賴近期查詢來估計未來符元的重要性，並隱含假設這些查詢可作為未來注意力模式的可靠代理。我們證明此假設在長時程推理中並不成立：某些解碼步驟會產生思維重訪符元（TRT），其會重新關注遙遠的先前上下文，例如在軌跡早期形成的解題計畫。透過系統性分析，我們發現對應於 TRT 的查詢在嵌入空間中會聚集成少數相似群組。基於此洞察，我們提出 BeaconKV，一種無需訓練的 KV 快取壓縮方法；其維護信標查詢——每個全域查詢叢集的精簡代表——以預測哪些 KV 對將被重新關注，而無需儲存完整的查詢歷史。在四個開源 LRM 與多樣的推理基準上，BeaconKV 通常優於現有壓縮方法，可達到最高 5.8 倍的記憶體縮減，同時幾乎維持完整快取準確率，並將吞吐量提升超過 4.3 倍。

English

Large Reasoning Models (LRMs) achieve superior problem-solving through extended Chain-of-Thought (CoT) generation, but the resulting key-value (KV) cache grows linearly with sequence length and creates severe memory bottlenecks, often exceeding GPU capacity for long reasoning traces. Existing KV cache compression methods rely on recent queries to estimate future token importance, implicitly assuming these serve as reliable proxies for future attention patterns. We demonstrate that this assumption fails in long-horizon reasoning: certain decoding steps generate Thought Revisiting Tokens (TRT) that re-attend to distant previous context, such as task-solving plans formulated early in the trace. Through systematic analysis, we discover that queries corresponding to the TRT cluster into a small number of similarity groups in the embedding space. Based on this insight, we propose BeaconKV, a training-free KV cache compression method that maintains beacon queries, compact representatives for each global query cluster, to anticipate which KV pairs will be revisited without storing the entire query history. Across four open-source LRMs and diverse reasoning benchmarks, BeaconKV generally outperforms existing compression methods, achieving up to 5.8times memory reduction while nearly preserving full cache accuracy and improving throughput by over 4.3times.