迷失於壓縮：抽取式提示詞壓縮器的受控跨語言審計

摘要

抽取式提示詞壓縮旨在透過移除低資訊 token 來降低 LLM 推論成本，而 LLMLingua-2 等學習式壓縮器在英文基準上報告了強勁的結果。然而，大多數其他語言已經在付出 token 溢價：相同內容所需的 token 數量比英文多 1.3 至 1.8 倍。我們要問的是，壓縮是否縮小還是擴大了這個差距。我們使用涵蓋五種文字系統的十種語言的完全平行資料，並在目標模型分詞器的預算匹配控制下，針對四種確定性基準方法審核了四種學習式壓縮器，涵蓋來自十家供應商的十一種目標模型（超過 25 萬次評估呼叫）。其中三種壓縮器以英文監督訓練（LLMLingua-2 XLM-R/mBERT；來自生產級 Headroom 技術棧的 Kompress-v2）；第四種 XProvence 則以多語言方式訓練。首先，遷移差距確實存在，跨目標模型與壓縮器骨幹皆可重現，且強烈依賴保留率：在 0.33 的保留率下，英文保留了 57-62% 的正規化上下文利用率，立陶宛文保留了 10-24%，而中文幾乎沒有保留，儘管中文的 token 溢價最小。其次，這個差距源於壓縮監督資料，而非架構。所有三種英文訓練的壓縮器都顯示出此差距，確定性方法沒有類似的差距，而多語言訓練的 XProvence v1 則沒有。其 v2 版本使用翻譯資料重新訓練，在激進閾值下毫無預警地清空了 92% 的中文上下文。第三，在更困難的長上下文設定中，激進的學習式壓縮在五種非英文語言中的三種中，將壓縮後上下文推向無上下文效用水準或更低。在五種受測語言中，先翻譯後壓縮的流程在三種語言中與原生壓縮相當或更好，且 token 成本約為一半。我們釋出所有程式碼、壓縮結果與模型輸出。在英文之外，安全的壓縮預算要小得多。

English

Extractive prompt compression promises to cut LLM inference costs by removing low-information tokens, and learned compressors such as LLMLingua-2 report strong results on English benchmarks. Most other languages already pay a token premium: the same content costs 1.3-1.8x more tokens than in English. We ask whether compression closes or widens this gap. Using fully parallel data in ten languages spanning five scripts, with controls budget-matched in the target model's tokenizer, we audit four learned compressors against four deterministic baselines, on eleven target models from ten vendors (over 250,000 evaluation calls). Three of the compressors are trained with English supervision (LLMLingua-2 XLM-R/mBERT; Kompress-v2 from the production Headroom stack); the fourth, XProvence, is trained multilingually. First, the transfer gap is real, replicates across target models and compressor backbones, and is strongly rate-dependent: at a 0.33 keep-rate English retains 57-62% of normalized context utilization while Lithuanian retains 10-24% and Chinese essentially none, despite Chinese having the smallest token premium. Second, the gap tracks compression supervision data, not architecture. All three English-trained compressors show it, deterministic methods show no comparable gap, and the multilingually trained XProvence v1 shows none. Its v2 release, retrained on translated data, empties 92% of Chinese contexts at its aggressive threshold without any warning. Third, in a harder long-context setting, aggressive learned compression drives compressed contexts to or below no-context utility in three of five non-English languages. A translate-then-compress pipeline matches or beats native compression at roughly half the token cost in three of five tested languages. We release all code, compressions, and model outputs. Safe compression budgets are much smaller outside English.