更深並非總是更好：通過置信層解碼減輕對齊代價

摘要

大語言模型（LLMs）的自迴歸生成通常從最後一層解碼，假設更深層的表示能產生更可靠的下一個令牌預測。我們透過揭示一個反覆出現的「猜測-精煉-擾動」動態來重新審視此假設：早期層形成粗略猜測，中間層精煉與推理相關的語義，而最後一層可能將這些精煉後的預測擾動為通用或對齊偏好標記。我們提出「自信解碼」（Confident Decoding），一種無需訓練的解碼策略，透過熵引導的保守反向搜索動態選擇最可靠的接近最後一層。我們進一步將層選擇理論形式化為一個最佳停止問題，證明在有限的投影噪聲與主導性的後期對齊擾動下，我們的搜索規則能過濾擾動，同時相對於理想精煉層的損失保持有界。在密集與專家混合（Mixture-of-Experts）LLMs上的實驗顯示，在具挑戰性的推理基準（包括GPQA-Diamond、Omni-MATH與HLE）上取得一致的提升，且無記憶體開銷，延遲增加低於2%。這些結果表明，動態繞過最後一層的擾動，能從已對齊的LLMs中釋放出更強的推理行為。

English

Autoregressive generation in large language models (LLMs) conventionally decodes from the final layer, assuming that deeper representations yield more reliable next-token predictions. We revisit this assumption by revealing a recurring Guess-Refine-Perturb dynamic: early layers form coarse guesses, intermediate layers refine reasoning-relevant semantics, and final layers can perturb these refined predictions toward generic or alignment-preferred tokens. We introduce Confident Decoding, a training-free decoding strategy that dynamically selects the most reliable near-final layer through entropy-guided conservative backward search. We further provide a theoretical formulation of layer selection as an optimal stopping problem, showing that under bounded projection noise and dominant late-stage alignment perturbation, our search rule filters perturbation while bounding the loss relative to the oracle refinement layer. Experiments across dense and Mixture-of-Experts LLMs demonstrate consistent gains on challenging reasoning benchmarks, including GPQA-Diamond, Omni-MATH, and HLE, with zero memory overhead and less than 2% latency increase. These results suggest dynamically bypassing final-layer perturbations can unlock stronger reasoning behavior from aligned LLMs.