原位反馈：多轮推理中引导大语言模型的新范式

摘要

大型语言模型（LLMs）在多轮推理场景中的研究日益增多，其中模型会根据用户提供的反馈迭代优化其输出。此类场景对于需要复杂推理的任务至关重要，然而现有的反馈范式通常依赖于发送新消息。LLMs在可靠整合这些反馈方面存在困难，导致改进效果不一致。在本研究中，我们引入了原位反馈这一新颖的交互范式，用户直接编辑LLM的先前响应，模型则基于这一修改后的响应生成修订版本。在多种推理密集型基准上的实证评估表明，原位反馈相较于传统的多轮反馈实现了更优的性能，同时减少了79.1%的token使用量。在受控环境中的补充分析进一步证实，原位反馈解决了多轮反馈的一个核心局限：模型往往无法精确地将反馈应用于响应中的错误部分，导致错误未被纠正，有时甚至会在原本正确的内容中引入新的错误。这些发现表明，原位反馈为引导LLMs在推理密集型任务中提供了更为自然且有效的机制。

English

Large language models (LLMs) are increasingly studied in the context of multi-turn reasoning, where models iteratively refine their outputs based on user-provided feedback. Such settings are crucial for tasks that require complex reasoning, yet existing feedback paradigms often rely on issuing new messages. LLMs struggle to integrate these reliably, leading to inconsistent improvements. In this work, we introduce in-place feedback, a novel interaction paradigm in which users directly edit an LLM's previous response, and the model conditions on this modified response to generate its revision. Empirical evaluations on diverse reasoning-intensive benchmarks reveal that in-place feedback achieves better performance than conventional multi-turn feedback while using 79.1% fewer tokens. Complementary analyses on controlled environments further demonstrate that in-place feedback resolves a core limitation of multi-turn feedback: models often fail to apply feedback precisely to erroneous parts of the response, leaving errors uncorrected and sometimes introducing new mistakes into previously correct content. These findings suggest that in-place feedback offers a more natural and effective mechanism for guiding LLMs in reasoning-intensive tasks.

原位反馈：多轮推理中引导大语言模型的新范式

In-Place Feedback: A New Paradigm for Guiding LLMs in Multi-Turn Reasoning

摘要

Support