潛在推理研究綜述

A Survey on Latent Reasoning

July 8, 2025

作者: Rui-Jie Zhu, Tianhao Peng, Tianhao Cheng, Xingwei Qu, Jinfa Huang, Dawei Zhu, Hao Wang, Kaiwen Xue, Xuanliang Zhang, Yong Shan, Tianle Cai, Taylor Kergan, Assel Kembay, Andrew Smith, Chenghua Lin, Binh Nguyen, Yuqi Pan, Yuhong Chou, Zefan Cai, Zhenhe Wu, Yongchi Zhao, Tianyu Liu, Jian Yang, Wangchunshu Zhou, Chujie Zheng, Chongxuan Li, Yuyin Zhou, Zhoujun Li, Zhaoxiang Zhang, Jiaheng Liu, Ge Zhang, Wenhao Huang, Jason Eshraghian

cs.AI

摘要

大型語言模型（LLMs）已展現出令人印象深刻的推理能力，尤其是在明確的思維鏈（CoT）推理指導下，這種推理能夠將中間步驟以語言形式表達出來。雖然CoT提升了模型的可解釋性和準確性，但其對自然語言推理的依賴限制了模型的表達帶寬。潛在推理則通過在模型的連續隱藏狀態中完全進行多步推理來解決這一瓶頸，從而消除了對標記級監督的需求。為了推動潛在推理研究的發展，本綜述提供了對這一新興領域的全面概述。我們首先探討了神經網絡層作為推理計算基礎的基礎性作用，強調了分層表示如何支持複雜的轉換。接著，我們探索了多種潛在推理方法，包括基於激活的遞歸、隱藏狀態傳播，以及壓縮或內化顯式推理軌跡的微調策略。最後，我們討論了高級範式，如通過掩碼擴散模型實現的無限深度潛在推理，這些模型能夠實現全局一致且可逆的推理過程。通過統一這些視角，我們旨在澄清潛在推理的概念圖景，並為LLM認知前沿的研究指明未來方向。相關的GitHub倉庫收集了最新的論文和代碼庫，可訪問：https://github.com/multimodal-art-projection/LatentCoT-Horizon/。

English

Large Language Models (LLMs) have demonstrated impressive reasoning capabilities, especially when guided by explicit chain-of-thought (CoT) reasoning that verbalizes intermediate steps. While CoT improves both interpretability and accuracy, its dependence on natural language reasoning limits the model's expressive bandwidth. Latent reasoning tackles this bottleneck by performing multi-step inference entirely in the model's continuous hidden state, eliminating token-level supervision. To advance latent reasoning research, this survey provides a comprehensive overview of the emerging field of latent reasoning. We begin by examining the foundational role of neural network layers as the computational substrate for reasoning, highlighting how hierarchical representations support complex transformations. Next, we explore diverse latent reasoning methodologies, including activation-based recurrence, hidden state propagation, and fine-tuning strategies that compress or internalize explicit reasoning traces. Finally, we discuss advanced paradigms such as infinite-depth latent reasoning via masked diffusion models, which enable globally consistent and reversible reasoning processes. By unifying these perspectives, we aim to clarify the conceptual landscape of latent reasoning and chart future directions for research at the frontier of LLM cognition. An associated GitHub repository collecting the latest papers and repos is available at: https://github.com/multimodal-art-projection/LatentCoT-Horizon/.