通过原生检索增强推理提升上下文保真度

摘要

大型语言模型（LLMs）在处理上下文一致性时常常面临挑战，在基于提供信息回答问题时，往往会产生不一致的答案。现有方法要么依赖昂贵的监督微调来在生成答案后提供证据，要么训练模型执行网络搜索，却未必能有效利用给定的上下文。我们提出了CARE，一种新颖的原生检索增强推理框架，它教导LLMs在推理过程中明确整合上下文证据，并利用模型自身的检索能力。我们的方法仅需少量标注的证据数据，通过策略性地在推理链中检索上下文标记，显著提升了检索准确性和答案生成性能。在多个现实世界和反事实问答基准上的广泛实验表明，我们的方法大幅超越了监督微调、传统的检索增强生成方法以及外部检索解决方案。这一工作标志着在使LLMs更准确、可靠且高效地执行知识密集型任务方面取得了根本性进展。

English

Large language models (LLMs) often struggle with context fidelity, producing inconsistent answers when responding to questions based on provided information. Existing approaches either rely on expensive supervised fine-tuning to generate evidence post-answer or train models to perform web searches without necessarily improving utilization of the given context. We propose CARE, a novel native retrieval-augmented reasoning framework that teaches LLMs to explicitly integrate in-context evidence within their reasoning process with the model's own retrieval capabilities. Our method requires limited labeled evidence data while significantly enhancing both retrieval accuracy and answer generation performance through strategically retrieved in-context tokens in the reasoning chain. Extensive experiments on multiple real-world and counterfactual QA benchmarks demonstrate that our approach substantially outperforms supervised fine-tuning, traditional retrieval-augmented generation methods, and external retrieval solutions. This work represents a fundamental advancement in making LLMs more accurate, reliable, and efficient for knowledge-intensive tasks.

通过原生检索增强推理提升上下文保真度

Improving Context Fidelity via Native Retrieval-Augmented Reasoning

摘要

Support