知曉何時放棄：診斷與訓練大型語言模型以中止徒勞推理

摘要

大型語言模型在超出其能力的任務上，會產生計算成本高昂卻語義空洞的推理，此現象造成可聽起來合理但內容錯誤的推導過程誤導使用者，衍生諸多風險。我們透過系統性分析來刻畫此一徒勞推理現象，揭示普遍存在的能力越界，以及能力與行為之間系統性的校準失準。最主要的失敗模式為虛假推理，其輸出表面上看似有效，實則含有細微錯誤，且此問題隨任務難度提升而加劇。為解決此問題，我們提出CaRL（能力對齊強化學習），透過獎勵塑形機制激勵模型拒絕作答而非進行徒勞推理，並輔以事後拒絕增強機制將失敗案例轉化為拒絕監督訊號，從而將模型行為與能力邊界對齊。實驗結果顯示，該方法在維持各任務難度下表現的同時，大幅減少了徒勞推理，有效實現能力對齊行為而不損及實用性。https://github.com/icip-cas/Knowing-When-to-Quit

English

Large language models generate computationally expensive yet semantically void reasoning on beyond-capability tasks, creating risks where plausible-sounding but incorrect derivations mislead users. We characterize this futile reasoning phenomenon through systematic analysis, revealing universal capability overreach and systematic miscalibration between capability and behavior. The dominant failure mode is specious reasoning, which outputs look superficially valid but contain subtle errors, escalating with task difficulty. To address this, we introduce CaRL (Capability-aligned Reinforcement Learning), which aligns model behavior with capability boundaries through reward shaping that incentivizes refusal over futile reasoning and hindsight refusal augmentation that converts failures into refusal supervision. Experiments demonstrate a substantial reduction in futile reasoning while preserving performance across task difficulties, effectively achieving capability-aligned behavior without sacrificing utility. https://github.com/icip-cas/Knowing-When-to-Quit