GradCuit：信用分配梯度流實現穩健且可解釋的測試時潛在推理

摘要

基於優化的潛在推理透過在測試時優化每個實例的連續狀態來提升大型語言模型的輸出，同時保持模型參數凍結。然而，現有方法通常透過解碼後的詞元將這些狀態與推理軌跡連結起來，使得序列層級的信用分配變得不直接，並模糊了潛在更新如何塑造後續推理。我們提出GradCuit（透過電路的梯度），它在選定的Transformer層中，於提示的隱藏表示與生成延續之間插入可優化的潛在狀態。因果自注意力機制為每個延續詞元的對數機率提供一條可微分的路徑，使其經由其餘Transformer區塊通向每個先前的潛在狀態，從而使得來自整個延續的獎勵加權梯度能夠直接分配給潛在狀態。在五個經指令微調的骨幹模型、三個推理基準測試和兩種答案格式中，GradCuit達到了64.5%的平均準確率，比思維鏈提示高出6.6個百分點，比最強的對比方法高出2.4個百分點。GradCuit也展現出更強的穩健性：在七種學習率設定下，它持續優於LatentSeek，同時將準確率的標準差從1.53降至0.82，甚至其隨機遊走變體也能與LatentSeek保持競爭力。在可解釋性方面，詞元層級的梯度歸因顯示，潛在影響集中在推理連接詞元上，而層分析則指出早期至中期的Transformer層是最有效的優化空間。透過直接從結果回饋中優化內部推理，GradCuit開闢了一條穩健且可解釋的測試時擴展新軸線，使大型語言模型能夠調整其推理方式，而非僅僅重新生成、抽樣或重新排序輸出。

English

Optimization-based latent reasoning improves large language model outputs by optimizing instance-specific continuous states at test time while keeping model parameters frozen. Existing methods, however, typically connect these states to the reasoning trajectory through decoded tokens, making sequence-level credit assignment indirect and obscuring how latent updates shape subsequent reasoning. We introduce GradCuit (gradient through circuit), which inserts optimizable latent states at a selected Transformer layer between the hidden representations of the prompt and the generated continuation. Causal self-attention provides every continuation-token log-probability with a differentiable path to every preceding latent state through the remaining Transformer blocks, enabling reward-weighted gradients from the entire continuation to be assigned directly to the latents. Across five instruction-tuned backbones, three reasoning benchmarks, and two answer formats, GradCuit achieves an average accuracy of 64.5%, outperforming chain-of-thought prompting by 6.6 percentage points and the strongest competing method by 2.4 points. GradCuit also demonstrates greater robustness: across seven learning-rate settings, it consistently outperforms LatentSeek while reducing the standard deviation of accuracy from 1.53 to 0.82, and even its random-walk variant remains competitive with LatentSeek. For interpretability, token-level gradient attribution reveals that latent influence concentrates on reasoning-connector tokens, while layer analysis identifies early-to-middle Transformer layers as the most effective optimization space. By directly optimizing internal reasoning from outcome feedback, GradCuit opens a new axis of robust and interpretable test-time scaling, where LLMs adapt how they reason rather than merely regenerate, sample, or rerank outputs.