相关但不完整：硬提示压缩中的范式级失败模式——指称悬空

摘要

硬提示压缩通过独立地对词元、句子或块进行评分，并在预算约束下保留得分最高的单元，来降低长上下文推理成本。我们发现了该过程的一个结构性缺陷：独立选择可能会拆分相互依赖的证据对，保留其中一个成员而删除另一个。当保留文本包含答案，而删除文本定义了解读该答案所需的实体时，我们称这一结果为指称悬空。在压缩率为0.30时，使用Qwen3-0.6B嵌入对连贯块进行排序的Beaver，在三个多跳问答数据集的桥接示例中，有34%-54%的示例答案路径不完整。在一个共享的HotpotQA桥接集合上，我们测试的全部六个硬压缩器均出现悬空现象，最高悬空率达60%；并且LongBench-v2单文档问答中的每篇文档都至少包含一个悬空引用。在使用Qwen3-8B评估的悬空示例上，重新插入缺失的支持段落并删除非支持段落以维持词元预算，可使准确率提升29-34个百分点（p < 0.0001），至少恢复了与保留两个支持段落的上下文之间差距的88%。更强的答案模型并不能弥补这一损失：在MuSiQue上，GPT-5.5在压缩上下文上的准确率比在保留两个支持段落的上下文上低8.8个百分点。最后，我们训练了一个精简分类器，根据被省略句子是否为解读保留文本所需对其进行排序，并在推理时无需支持标注即可重新插入排名靠前的候选句子。在HotpotQA上使用Qwen3-8B时，这种自动恢复将准确率提升了4.7个百分点，而压缩率仅从0.30变为0.31。硬压缩器应同时优化相关性和指称完整性。

English

Hard prompt compression reduces long-context inference cost by independently scoring tokens, sentences, or chunks and retaining the highest-scoring units under a budget. We identify a structural failure in this procedure: independent selection can split dependent evidence pairs, retaining one member while deleting the other. When retained text contains an answer but deleted text defines the entity needed to interpret it, we call the result referential dangling. At a compression ratio of 0.30, Beaver, which ranks coherent chunks using Qwen3-0.6B embeddings, leaves the answer path incomplete in 34-54% of bridge examples across three multi-hop question answering datasets. On a shared HotpotQA bridge set, all six hard compressors we test exhibit dangling at rates up to 60%, and every document in LongBench-v2 Single-Document QA contains at least one dangling reference. On dangling examples evaluated with Qwen3-8B, reinserting the missing supporting paragraph while removing nonsupporting paragraphs to maintain the token budget improves accuracy by 29-34 percentage points (p < 0.0001), recovering at least 88% of the gap to contexts retaining both supporting paragraphs. Stronger answer models do not absorb the loss: on MuSiQue, GPT-5.5 is 8.8 points less accurate on compressed contexts than on contexts retaining both supporting paragraphs. Finally, we train a compact classifier to rank omitted sentences by whether they are needed to interpret retained text and reinsert the top-ranked candidates without support annotations at inference. On HotpotQA with Qwen3-8B, this automatic restoration improves accuracy by 4.7 points while changing the compression ratio only from 0.30 to 0.31. Hard compressors should optimize both relevance and referential completeness.