相關但殘缺：指涉懸置作為硬提示壓縮中的範式層級失效模式

摘要

硬性提示壓縮透過獨立評分詞元、句子或區塊，並在預算限制下保留得分最高的單元，以降低長上下文推論成本。我們在此程序中發現一個結構性缺陷：獨立選擇可能拆散相依的證據對，保留其中一個成員而刪除另一個。當保留的文字包含答案，但被刪除的文字定義了解讀答案所需的實體時，我們稱此結果為指涉懸置。在壓縮比率為0.30時，使用Qwen3-0.6B嵌入對連貫區塊進行排序的Beaver，在三個多跳式問答資料集中，有34-54%的橋接範例留下不完整的答案路徑。在共享的HotpotQA橋接集合上，我們測試的所有六種硬性壓縮器都表現出高達60%的懸置率，且LongBench-v2單文件問答中的每份文件都包含至少一個懸置指涉。在使用Qwen3-8B評估的懸置範例中，重新插入缺失的支持段落並移除不支持段落以維持詞元預算，可將準確率提升29-34個百分點（p < 0.0001），並恢復至少88%與保留兩個支持段落的上下文之間的差距。更強的答案模型並未吸收此損失：在MuSiQue上，GPT-5.5在壓縮上下文上的準確率比保留兩個支持段落的上下文低8.8個百分點。最後，我們訓練一個輕量級分類器，依據被省略的句子是否為解讀保留文字所需來排序，並在推論時無需支持註釋即可重新插入排名最高的候選句子。在HotpotQA搭配Qwen3-8B的實驗中，這種自動還原將準確率提升了4.7個百分點，而壓縮比率僅從0.30變化至0.31。硬性壓縮器應同時最佳化相關性與指涉完整性。

English

Hard prompt compression reduces long-context inference cost by independently scoring tokens, sentences, or chunks and retaining the highest-scoring units under a budget. We identify a structural failure in this procedure: independent selection can split dependent evidence pairs, retaining one member while deleting the other. When retained text contains an answer but deleted text defines the entity needed to interpret it, we call the result referential dangling. At a compression ratio of 0.30, Beaver, which ranks coherent chunks using Qwen3-0.6B embeddings, leaves the answer path incomplete in 34-54% of bridge examples across three multi-hop question answering datasets. On a shared HotpotQA bridge set, all six hard compressors we test exhibit dangling at rates up to 60%, and every document in LongBench-v2 Single-Document QA contains at least one dangling reference. On dangling examples evaluated with Qwen3-8B, reinserting the missing supporting paragraph while removing nonsupporting paragraphs to maintain the token budget improves accuracy by 29-34 percentage points (p < 0.0001), recovering at least 88% of the gap to contexts retaining both supporting paragraphs. Stronger answer models do not absorb the loss: on MuSiQue, GPT-5.5 is 8.8 points less accurate on compressed contexts than on contexts retaining both supporting paragraphs. Finally, we train a compact classifier to rank omitted sentences by whether they are needed to interpret retained text and reinsert the top-ranked candidates without support annotations at inference. On HotpotQA with Qwen3-8B, this automatic restoration improves accuracy by 4.7 points while changing the compression ratio only from 0.30 to 0.31. Hard compressors should optimize both relevance and referential completeness.