大语言模型中的元认知：基础、进展与机遇

摘要

元认知是智力的基础组成部分，对有效学习、问题解决、决策制定、沟通等方面至关重要。近年来，它逐渐被视为构建强大、透明人工智能系统的核心要素。然而，尽管大语言模型（LLM）在多种现实任务中取得了显著进展，但目前尚不清楚它们何时、以何种方式以及在多大程度上能够展现或具备有效的元认知能力，也不清楚如何利用这些能力来提升AI系统的基础能力、可靠性和智能水平。本文通过首次系统梳理LLM元认知研究现状，填补了这一空白。我们分析并分类了这一新兴领域的格局，总结了最新技术进展，包括用于测量和评估LLM元认知能力的方法与基准、激发、改进和应用LLM元认知的技术，以及当前研究的发现与启示。我们还讨论了应用场景、开放问题与挑战，以及未来工作的潜在方向。旨在提供该主题的详尽、最新综述，并推动有意义的研究与讨论。相关文献列表可参见https://github.com/yale-nlp/LLM-Metacognition。

English

Metacognition is a foundational component of intelligence critical to effective learning, problem solving, decision-making, communication, and more. In recent years, it has become increasingly recognized as a cornerstone of capable, transparent AI systems. Yet while LLMs have made significant progress across diverse real-world tasks, it is not yet clear when, how, or to what extent they can exhibit or be endowed with effective metacognitive abilities, nor how such abilities can be adapted to advance the fundamental capabilities, reliability, and intelligence of AI systems. This paper bridges this gap by presenting the first comprehensive overview of the current state of knowledge on metacognition for LLMs. We analyze and taxonomize the landscape of this emerging field and summarize recent technical advancements, including methods and benchmarks to measure and evaluate LLMs' metacognitive abilities, techniques to elicit, improve, and apply metacognition in LLMs, and findings and implications of ongoing research. We also discuss applications, open questions and challenges, and promising directions for future work. Our aim is to provide a detailed and up-to-date review of this topic and stimulate meaningful research and discussion. An organized list of papers can be found at https://github.com/yale-nlp/LLM-Metacognition.