KV-CoRE：基准测试LLM中KV缓存的数据依赖性低秩可压缩性

摘要

大型語言模型依賴鍵值快取（kv-cache）來避免自迴歸解碼過程中的冗餘計算，但隨著上下文長度增加，讀寫快取會迅速飽和GPU記憶體頻寬。現有研究雖已探索KV快取壓縮技術，但多數方法忽略了kv-cache的數據依賴特性及其在不同層級的動態變化。本文提出KV-CoRE（基於秩評估的KV快取可壓縮性量化方法），這是一種基於奇異值分解（SVD）的技術，可量化kv-cache的數據依賴型低秩可壓縮性。該方法通過弗羅貝尼烏斯範數計算最優低秩近似，且無需梯度計算並支持增量處理，能實現高效的數據集級別分層評估。基於此方法，我們分析了涵蓋五個英語領域和十六種語言的多個模型與數據集，揭示了可壓縮性與模型架構、訓練數據及語言覆蓋範圍的系統性關聯規律。在分析過程中，我們採用歸一化有效秩作為可壓縮性度量指標，並證明其與壓縮下的性能衰減存在強相關性。本研究建立了首個大規模LLM的kv-cache可壓縮性基準評估框架，為動態感知數據的壓縮技術及以數據為中心的模型開發提供了新視角。

English

Large language models rely on kv-caches to avoid redundant computation during autoregressive decoding, but as context length grows, reading and writing the cache can quickly saturate GPU memory bandwidth. Recent work has explored KV-cache compression, yet most approaches neglect the data-dependent nature of kv-caches and their variation across layers. We introduce KV-CoRE KV-cache Compressibility by Rank Evaluation), an SVD-based method for quantifying the data-dependent low-rank compressibility of kv-caches. KV-CoRE computes the optimal low-rank approximation under the Frobenius norm and, being gradient-free and incremental, enables efficient dataset-level, layer-wise evaluation. Using this method, we analyze multiple models and datasets spanning five English domains and sixteen languages, uncovering systematic patterns that link compressibility to model architecture, training data, and language coverage. As part of this analysis, we employ the Normalized Effective Rank as a metric of compressibility and show that it correlates strongly with performance degradation under compression. Our study establishes a principled evaluation framework and the first large-scale benchmark of kv-cache compressibility in LLMs, offering insights for dynamic, data-aware compression and data-centric model development.

KV-CoRE：基准测试LLM中KV缓存的数据依赖性低秩可压缩性

KV-CoRE: Benchmarking Data-Dependent Low-Rank Compressibility of KV-Caches in LLMs

摘要

Support