通过动态笔记撰写增强大语言模型在复杂问答中的推理能力

Augmenting LLM Reasoning with Dynamic Notes Writing for Complex QA

May 22, 2025

作者: Rishabh Maheshwary, Masoud Hashemi, Khyati Mahajan, Shiva Krishna Reddy Malay, Sai Rajeswar, Sathwik Tejaswi Madhusudhan, Spandana Gella, Vikas Yadav

cs.AI

摘要

在多跳问答任务中，迭代式检索增强生成（RAG）面临长上下文和无关信息累积的挑战。这些问题阻碍了模型处理并推理检索内容的能力，限制了其性能表现。尽管近期方法聚焦于压缩检索信息，但它们要么仅限于单轮RAG，要么需要微调，或在迭代RAG中缺乏可扩展性。为应对这些挑战，我们提出了“笔记撰写”方法，该方法在每一步从检索文档中生成简洁且相关的笔记，从而减少噪声，仅保留关键信息。这间接增加了大型语言模型（LLMs）的有效上下文长度，使其在处理更大规模输入文本时能更有效地进行推理和规划。“笔记撰写”与框架无关，可集成于不同的迭代RAG方法中。我们通过在两种模型和四个评估数据集上应用三种迭代RAG方法，验证了其有效性。结果显示，“笔记撰写”平均整体提升了15.6个百分点，且输出标记数仅略有增加。

English

Iterative RAG for multi-hop question answering faces challenges with lengthy contexts and the buildup of irrelevant information. This hinders a model's capacity to process and reason over retrieved content and limits performance. While recent methods focus on compressing retrieved information, they are either restricted to single-round RAG, require finetuning or lack scalability in iterative RAG. To address these challenges, we propose Notes Writing, a method that generates concise and relevant notes from retrieved documents at each step, thereby reducing noise and retaining only essential information. This indirectly increases the effective context length of Large Language Models (LLMs), enabling them to reason and plan more effectively while processing larger volumes of input text. Notes Writing is framework agnostic and can be integrated with different iterative RAG methods. We demonstrate its effectiveness with three iterative RAG methods, across two models and four evaluation datasets. Notes writing yields an average improvement of 15.6 percentage points overall, with minimal increase in output tokens.