當信心誤導：擴散語言模型中的後綴錨定與錨點鄰近信心調節

摘要

擴散語言模型通過迭代對遮罩詞元序列進行去噪來解碼文本，因此選擇哪些位置進行解碼成為一項關鍵的推論階段決策。大多數無需訓練的解碼策略使用模型信心來選擇位置，假設高信心位置已準備好被解碼。在本研究中，我們通過探討信心何時誤導完全非自迴歸解碼，重新審視此假設。結束符記號可能獲得高信心並導致生成不完整；插入後綴錨點可緩解此問題，但會在錨點附近引發局部過度信心，導致錨點相鄰詞元過早被解碼。為解決這些問題，我們提出後綴錨點信心調節方法，這是一種簡單且無需訓練的方法，通過插入短後綴錨點來促進回應完成，並根據解碼進度調節錨點附近的信心。此方法保留了後綴錨點提升回應完成度的優點，同時減少錨點相鄰詞元的過早解碼。在純文本推理、視覺語言推理與程式碼生成基準測試中，我們的方法持續提升了基於信心的完全非自迴歸解碼表現，優於顯式的結束符記號抑制策略，並保留了完全非自迴歸生成的平行解碼優勢。

English

Diffusion language models decode text by iteratively denoising masked token sequences, making the choice of which positions to decode a central inference-time decision. Most training-free decoding strategies use model confidence for position selection, assuming that high-confidence positions are ready to be decoded. In this work, we revisit this assumption by studying when confidence misleads fully non-autoregressive (fully non-AR) decoding. EOT tokens can receive high confidence and cause incomplete generation; inserting a suffix anchor can mitigate this issue but introduces local overconfidence near the anchor, causing anchor-adjacent tokens to be decoded too early. To address these issues, we propose Suffix-Anchored Confidence Modulation, a simple training-free method that inserts a short suffix anchor to encourage response completion and modulates confidence near the anchor according to decoding progress. This preserves the response-completion benefit of suffix anchoring while reducing premature decoding of anchor-adjacent tokens. Across text-only reasoning, vision-language reasoning, and code-generation benchmarks, our method consistently improves confidence-based fully non-AR decoding, outperforms explicit EOT suppression, and preserves the parallel decoding advantage of fully non-AR generation.