基于早期知识对齐的多跳推理

摘要

检索增强生成（RAG）已成为大语言模型（LLM）处理需要领域专业知识或最新信息的知识密集型查询的重要范式。为应对单步检索难以解决的复杂多跳问题，研究者提出了结合强化学习的迭代式RAG方法。然而，现有迭代RAG系统通常在规划问题分解时未充分利用检索语料库的信息，导致检索效率低下，推理链错误累积进而影响性能。本文提出早期知识对齐（EKA）模块，该模块通过在迭代RAG系统中引入上下文相关的检索知识，使LLM在规划前与检索集实现对齐。在六个标准RAG数据集上的实验表明，EKA通过构建更坚实的推理基础，显著提升检索精度、减少错误传导，同时改善性能与效率。从信息熵角度的分析证明，早期知识的引入能减少推理过程中不必要的探索，使模型更聚焦于相关信息子集。此外，EKA作为一种无需训练的通用推理策略，可无缝扩展至大模型。跨数据集与检索语料库的泛化测试验证了方法的鲁棒性。总体而言，EKA在推进迭代RAG技术前沿的同时，揭示了强化学习增强框架中结构化推理与高效探索的关键交互机制。代码已发布于https://github.com/yxzwang/EarlyKnowledgeAlignment{Github}。

English

Retrieval-Augmented Generation (RAG) has emerged as a powerful paradigm for Large Language Models (LLMs) to address knowledge-intensive queries requiring domain-specific or up-to-date information. To handle complex multi-hop questions that are challenging for single-step retrieval, iterative RAG approaches incorporating reinforcement learning have been proposed. However, existing iterative RAG systems typically plan to decompose questions without leveraging information about the available retrieval corpus, leading to inefficient retrieval and reasoning chains that cascade into suboptimal performance. In this paper, we introduce Early Knowledge Alignment (EKA), a simple but effective module that aligns LLMs with retrieval set before planning in iterative RAG systems with contextually relevant retrieved knowledge. Extensive experiments on six standard RAG datasets demonstrate that by establishing a stronger reasoning foundation, EKA significantly improves retrieval precision, reduces cascading errors, and enhances both performance and efficiency. Our analysis from an entropy perspective demonstrate that incorporating early knowledge reduces unnecessary exploration during the reasoning process, enabling the model to focus more effectively on relevant information subsets. Moreover, EKA proves effective as a versatile, training-free inference strategy that scales seamlessly to large models. Generalization tests across diverse datasets and retrieval corpora confirm the robustness of our approach. Overall, EKA advances the state-of-the-art in iterative RAG systems while illuminating the critical interplay between structured reasoning and efficient exploration in reinforcement learning-augmented frameworks. The code is released at https://github.com/yxzwang/EarlyKnowledgeAlignment{Github}.

基于早期知识对齐的多跳推理

Multi-hop Reasoning via Early Knowledge Alignment

摘要

Support