潜在推理研究综述

A Survey on Latent Reasoning

July 8, 2025

作者: Rui-Jie Zhu, Tianhao Peng, Tianhao Cheng, Xingwei Qu, Jinfa Huang, Dawei Zhu, Hao Wang, Kaiwen Xue, Xuanliang Zhang, Yong Shan, Tianle Cai, Taylor Kergan, Assel Kembay, Andrew Smith, Chenghua Lin, Binh Nguyen, Yuqi Pan, Yuhong Chou, Zefan Cai, Zhenhe Wu, Yongchi Zhao, Tianyu Liu, Jian Yang, Wangchunshu Zhou, Chujie Zheng, Chongxuan Li, Yuyin Zhou, Zhoujun Li, Zhaoxiang Zhang, Jiaheng Liu, Ge Zhang, Wenhao Huang, Jason Eshraghian

cs.AI

摘要

大型语言模型（LLMs）已展现出卓越的推理能力，尤其是在显式思维链（CoT）推理的引导下，通过语言化中间步骤来增强理解。尽管CoT提升了模型的可解释性与准确性，其对自然语言推理的依赖却限制了模型的表达带宽。潜在推理通过完全在模型的连续隐藏状态中进行多步推理，解决了这一瓶颈，无需依赖词元级别的监督。为了推动潜在推理研究的发展，本综述全面概述了这一新兴领域。我们首先探讨了神经网络层作为推理计算基础的核心作用，强调层次化表示如何支持复杂的转换过程。接着，我们深入研究了多种潜在推理方法，包括基于激活的递归、隐藏状态传播，以及通过压缩或内化显式推理轨迹的微调策略。最后，我们讨论了诸如通过掩码扩散模型实现的无限深度潜在推理等高级范式，这些范式支持全局一致且可逆的推理过程。通过整合这些视角，我们旨在厘清潜在推理的概念框架，并为LLM认知前沿研究指明未来方向。相关GitHub仓库汇集了最新论文与资源，访问地址为：https://github.com/multimodal-art-projection/LatentCoT-Horizon/。

English

Large Language Models (LLMs) have demonstrated impressive reasoning capabilities, especially when guided by explicit chain-of-thought (CoT) reasoning that verbalizes intermediate steps. While CoT improves both interpretability and accuracy, its dependence on natural language reasoning limits the model's expressive bandwidth. Latent reasoning tackles this bottleneck by performing multi-step inference entirely in the model's continuous hidden state, eliminating token-level supervision. To advance latent reasoning research, this survey provides a comprehensive overview of the emerging field of latent reasoning. We begin by examining the foundational role of neural network layers as the computational substrate for reasoning, highlighting how hierarchical representations support complex transformations. Next, we explore diverse latent reasoning methodologies, including activation-based recurrence, hidden state propagation, and fine-tuning strategies that compress or internalize explicit reasoning traces. Finally, we discuss advanced paradigms such as infinite-depth latent reasoning via masked diffusion models, which enable globally consistent and reversible reasoning processes. By unifying these perspectives, we aim to clarify the conceptual landscape of latent reasoning and chart future directions for research at the frontier of LLM cognition. An associated GitHub repository collecting the latest papers and repos is available at: https://github.com/multimodal-art-projection/LatentCoT-Horizon/.