透過原生檢索增強推理提升上下文保真度

摘要

大型語言模型（LLMs）在處理上下文一致性時常面臨挑戰，在基於提供的信息回答問題時，往往會產生不一致的答案。現有的方法要么依賴於昂貴的監督微調來在回答後生成證據，要么訓練模型進行網絡搜索，而不一定提高對給定上下文的利用效率。我們提出了CARE，一種新穎的原生檢索增強推理框架，該框架教導LLMs在推理過程中明確整合上下文證據，並利用模型自身的檢索能力。我們的方法僅需有限的標記證據數據，同時通過在推理鏈中策略性地檢索上下文標記，顯著提升了檢索準確性和答案生成性能。在多個現實世界和反事實問答基準上的廣泛實驗表明，我們的方法大幅超越了監督微調、傳統的檢索增強生成方法以及外部檢索解決方案。這項工作代表了在使LLMs更準確、可靠和高效地執行知識密集型任務方面的一個根本性進步。

English

Large language models (LLMs) often struggle with context fidelity, producing inconsistent answers when responding to questions based on provided information. Existing approaches either rely on expensive supervised fine-tuning to generate evidence post-answer or train models to perform web searches without necessarily improving utilization of the given context. We propose CARE, a novel native retrieval-augmented reasoning framework that teaches LLMs to explicitly integrate in-context evidence within their reasoning process with the model's own retrieval capabilities. Our method requires limited labeled evidence data while significantly enhancing both retrieval accuracy and answer generation performance through strategically retrieved in-context tokens in the reasoning chain. Extensive experiments on multiple real-world and counterfactual QA benchmarks demonstrate that our approach substantially outperforms supervised fine-tuning, traditional retrieval-augmented generation methods, and external retrieval solutions. This work represents a fundamental advancement in making LLMs more accurate, reliable, and efficient for knowledge-intensive tasks.

透過原生檢索增強推理提升上下文保真度

Improving Context Fidelity via Native Retrieval-Augmented Reasoning

摘要

Support