駕馭推理經濟性：大型語言模型高效推理綜述

Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for Large Language Models

March 31, 2025

作者: Rui Wang, Hongru Wang, Boyang Xue, Jianhui Pang, Shudong Liu, Yi Chen, Jiahao Qiu, Derek Fai Wong, Heng Ji, Kam-Fai Wong

cs.AI

摘要

近年來，大型語言模型（LLMs）的進步顯著提升了其執行複雜推理任務的能力，從快速直覺的思維（系統1）過渡到緩慢深入的推理（系統2）。雖然系統2的推理提高了任務的準確性，但由於其緩慢的思維特性以及低效或不必要的推理行為，往往會帶來巨大的計算成本。相比之下，系統1的推理在計算上更為高效，但會導致次優的表現。因此，在性能（收益）與計算成本（預算）之間取得平衡至關重要，這便催生了推理經濟性的概念。在本調查中，我們全面分析了LLMs在訓練後和測試時推理階段的推理經濟性，涵蓋了：i）推理低效的原因，ii）不同推理模式的行為分析，以及iii）實現推理經濟性的潛在解決方案。通過提供可操作的見解並強調開放性挑戰，我們旨在揭示提升LLMs推理經濟性的策略，從而為這一不斷發展領域的研究提供寶貴資源。我們還提供了一個公共存儲庫，以持續追蹤這一快速發展領域的最新進展。

English

Recent advancements in Large Language Models (LLMs) have significantly enhanced their ability to perform complex reasoning tasks, transitioning from fast and intuitive thinking (System 1) to slow and deep reasoning (System 2). While System 2 reasoning improves task accuracy, it often incurs substantial computational costs due to its slow thinking nature and inefficient or unnecessary reasoning behaviors. In contrast, System 1 reasoning is computationally efficient but leads to suboptimal performance. Consequently, it is critical to balance the trade-off between performance (benefits) and computational costs (budgets), giving rise to the concept of reasoning economy. In this survey, we provide a comprehensive analysis of reasoning economy in both the post-training and test-time inference stages of LLMs, encompassing i) the cause of reasoning inefficiency, ii) behavior analysis of different reasoning patterns, and iii) potential solutions to achieve reasoning economy. By offering actionable insights and highlighting open challenges, we aim to shed light on strategies for improving the reasoning economy of LLMs, thereby serving as a valuable resource for advancing research in this evolving area. We also provide a public repository to continually track developments in this fast-evolving field.