当置信度误导时：扩散语言模型的后缀锚定与锚接近似置信度调制

摘要

扩散语言模型通过对掩码标记序列进行迭代去噪来解码文本，因此解码位置的选择成为推理阶段的核心决策。大多数无需训练的解码策略利用模型置信度进行位置选择，假设高置信度位置已准备就绪可被解码。本文通过研究置信度何时误导完全非自回归解码，重新审视了这一假设。序列结束标记可能获得高置信度并导致生成不完整；插入后缀锚点可缓解此问题，但会在锚点附近引发局部过度置信，导致锚点相邻标记过早起解码。为解决这些问题，我们提出后缀锚点置信度调制——一种简单的免训练方法，通过插入短后缀锚点促进响应完整性，并根据解码进度对锚点附近置信度进行调制。该方法在保持后缀锚点对响应完整性优势的同时，减少了锚点相邻标记的过早解码。在纯文本推理、视觉语言推理和代码生成基准测试中，我们的方法持续提升了基于置信度的完全非自回归解码性能，优于显式的EOT抑制策略，并保留了完全非自回归生成的并行解码优势。

English

Diffusion language models decode text by iteratively denoising masked token sequences, making the choice of which positions to decode a central inference-time decision. Most training-free decoding strategies use model confidence for position selection, assuming that high-confidence positions are ready to be decoded. In this work, we revisit this assumption by studying when confidence misleads fully non-autoregressive (fully non-AR) decoding. EOT tokens can receive high confidence and cause incomplete generation; inserting a suffix anchor can mitigate this issue but introduces local overconfidence near the anchor, causing anchor-adjacent tokens to be decoded too early. To address these issues, we propose Suffix-Anchored Confidence Modulation, a simple training-free method that inserts a short suffix anchor to encourage response completion and modulates confidence near the anchor according to decoding progress. This preserves the response-completion benefit of suffix anchoring while reducing premature decoding of anchor-adjacent tokens. Across text-only reasoning, vision-language reasoning, and code-generation benchmarks, our method consistently improves confidence-based fully non-AR decoding, outperforms explicit EOT suppression, and preserves the parallel decoding advantage of fully non-AR generation.