透過動態筆記撰寫增強大型語言模型的推理能力以應對複雜問答

Augmenting LLM Reasoning with Dynamic Notes Writing for Complex QA

May 22, 2025

作者: Rishabh Maheshwary, Masoud Hashemi, Khyati Mahajan, Shiva Krishna Reddy Malay, Sai Rajeswar, Sathwik Tejaswi Madhusudhan, Spandana Gella, Vikas Yadav

cs.AI

摘要

迭代式檢索增強生成（RAG）在多跳問答任務中面臨著長篇上下文和無關信息積累的挑戰。這阻礙了模型處理和推理檢索內容的能力，並限制了其表現。儘管近期方法專注於壓縮檢索信息，但它們要么僅適用於單輪RAG，要么需要微調，或在迭代RAG中缺乏可擴展性。為應對這些挑戰，我們提出了筆記撰寫法，該方法在每一步從檢索文檔中生成簡明且相關的筆記，從而減少噪音並僅保留關鍵信息。這間接增加了大型語言模型（LLMs）的有效上下文長度，使其在處理更大規模輸入文本時能更有效地進行推理和規劃。筆記撰寫法與框架無關，可與不同的迭代RAG方法集成。我們通過三種迭代RAG方法、兩種模型和四個評估數據集展示了其有效性。筆記撰寫法總體平均提升了15.6個百分點，且輸出標記的增加極小。

English

Iterative RAG for multi-hop question answering faces challenges with lengthy contexts and the buildup of irrelevant information. This hinders a model's capacity to process and reason over retrieved content and limits performance. While recent methods focus on compressing retrieved information, they are either restricted to single-round RAG, require finetuning or lack scalability in iterative RAG. To address these challenges, we propose Notes Writing, a method that generates concise and relevant notes from retrieved documents at each step, thereby reducing noise and retaining only essential information. This indirectly increases the effective context length of Large Language Models (LLMs), enabling them to reason and plan more effectively while processing larger volumes of input text. Notes Writing is framework agnostic and can be integrated with different iterative RAG methods. We demonstrate its effectiveness with three iterative RAG methods, across two models and four evaluation datasets. Notes writing yields an average improvement of 15.6 percentage points overall, with minimal increase in output tokens.