大型語言模型中的後設認知：基礎、進展與機遇

摘要

後設認知是智慧的基本組成部分，對於有效學習、問題解決、決策制定、溝通等方面至關重要。近年來，它越來越被視為具備能力且透明的AI系統的基石。然而，儘管大型語言模型在各種現實任務中取得了顯著進展，但目前仍不清楚它們何時、如何或在何種程度上能夠展現或具備有效的後設認知能力，也不清楚如何調整這些能力以提升AI系統的基本功能、可靠性與智慧。本文透過提供關於大型語言模型後設認知知識現狀的首份全面綜述來填補這一缺口。我們分析並分類這一新興領域的版圖，總結近期的技術進展，包括衡量與評估大型語言模型後設認知能力的方法與基準、引發、改善及應用大型語言模型中後設認知的技術，以及當前研究的發現與啟示。我們也討論了應用、開放性問題與挑戰，以及未來工作的有前景方向。我們的目標是提供關於此主題的詳細且最新綜述，並激發有意義的研究與討論。相關論文的有組織列表可見於 https://github.com/yale-nlp/LLM-Metacognition。

English

Metacognition is a foundational component of intelligence critical to effective learning, problem solving, decision-making, communication, and more. In recent years, it has become increasingly recognized as a cornerstone of capable, transparent AI systems. Yet while LLMs have made significant progress across diverse real-world tasks, it is not yet clear when, how, or to what extent they can exhibit or be endowed with effective metacognitive abilities, nor how such abilities can be adapted to advance the fundamental capabilities, reliability, and intelligence of AI systems. This paper bridges this gap by presenting the first comprehensive overview of the current state of knowledge on metacognition for LLMs. We analyze and taxonomize the landscape of this emerging field and summarize recent technical advancements, including methods and benchmarks to measure and evaluate LLMs' metacognitive abilities, techniques to elicit, improve, and apply metacognition in LLMs, and findings and implications of ongoing research. We also discuss applications, open questions and challenges, and promising directions for future work. Our aim is to provide a detailed and up-to-date review of this topic and stimulate meaningful research and discussion. An organized list of papers can be found at https://github.com/yale-nlp/LLM-Metacognition.