CRITIC：大型語言模型可以通過工具互動式批評進行自我校正。

摘要

近期大型語言模型（LLMs）的發展令人印象深刻。然而，這些模型有時會出現不一致和問題行為，例如幻覺事實、生成有缺陷的程式碼，或創造冒犯性和有毒內容。與這些模型不同，人類通常利用外部工具來交叉檢查和完善其初始內容，例如使用搜索引擎進行事實核查，或使用程式碼解釋器進行除錯。受到這一觀察的啟發，我們引入了一個名為CRITIC的框架，允許LLMs（本質上是“黑盒子”）驗證並逐步修正其輸出，類似於人類與工具的互動方式。更具體地說，從初始輸出開始，CRITIC與適當的工具互動，評估文本的某些方面，然後根據在此驗證過程中獲得的反饋修訂輸出。包括自由形式問答、數學程序合成和毒性減少在內的全面評估顯示，CRITIC持續增強了LLMs的性能。與此同時，我們的研究凸顯了外部反饋在促進LLMs持續自我改進中的至關重要性。

English

Recent developments in large language models (LLMs) have been impressive. However, these models sometimes show inconsistencies and problematic behavior, such as hallucinating facts, generating flawed code, or creating offensive and toxic content. Unlike these models, humans typically utilize external tools to cross-check and refine their initial content, like using a search engine for fact-checking, or a code interpreter for debugging. Inspired by this observation, we introduce a framework called CRITIC that allows LLMs, which are essentially "black boxes" to validate and progressively amend their own outputs in a manner similar to human interaction with tools. More specifically, starting with an initial output, CRITIC interacts with appropriate tools to evaluate certain aspects of the text, and then revises the output based on the feedback obtained during this validation process. Comprehensive evaluations involving free-form question answering, mathematical program synthesis, and toxicity reduction demonstrate that CRITIC consistently enhances the performance of LLMs. Meanwhile, our research highlights the crucial importance of external feedback in promoting the ongoing self-improvement of LLMs.

CRITIC：大型語言模型可以通過工具互動式批評進行自我校正。

CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing

摘要

Support