基于早期知识对齐的多跳推理

摘要

檢索增強生成（RAG）已成為大型語言模型（LLM）處理需要領域專屬或最新知識的密集型查詢的重要範式。為應對單步檢索難以解決的複雜多跳問題，學界提出了結合強化學習的迭代式RAG方法。然而，現有迭代RAG系統通常僅規劃問題分解策略，而未利用檢索語料庫的可用信息，導致檢索效率低下且推理鏈會引發次優性能級聯效應。本文提出早期知識對齊（EKA）模塊，該模塊通過在迭代RAG系統規劃前將LLM與上下文相關的檢索知識集進行對齊，其設計簡潔卻高效。在六個標準RAG數據集上的大量實驗表明，EKA通過建立更堅實的推理基礎，顯著提升檢索精度、減少級聯誤差，並同步改善性能與效率。從熵視角進行的分析證實，早期知識的引入能減少推理過程中不必要的探索，使模型更聚焦於相關信息子集。此外，EKA作為一種無需訓練的通用推理策略，可無縫擴展至大型模型。跨數據集與檢索語料庫的泛化測試驗證了該方法的魯棒性。總體而言，EKA在推進迭代RAG技術前沿的同時，揭示了強化學習增強框架中結構化推理與高效探索的關鍵互動機制。代碼已開源於：https://github.com/yxzwang/EarlyKnowledgeAlignment{Github}。

English

Retrieval-Augmented Generation (RAG) has emerged as a powerful paradigm for Large Language Models (LLMs) to address knowledge-intensive queries requiring domain-specific or up-to-date information. To handle complex multi-hop questions that are challenging for single-step retrieval, iterative RAG approaches incorporating reinforcement learning have been proposed. However, existing iterative RAG systems typically plan to decompose questions without leveraging information about the available retrieval corpus, leading to inefficient retrieval and reasoning chains that cascade into suboptimal performance. In this paper, we introduce Early Knowledge Alignment (EKA), a simple but effective module that aligns LLMs with retrieval set before planning in iterative RAG systems with contextually relevant retrieved knowledge. Extensive experiments on six standard RAG datasets demonstrate that by establishing a stronger reasoning foundation, EKA significantly improves retrieval precision, reduces cascading errors, and enhances both performance and efficiency. Our analysis from an entropy perspective demonstrate that incorporating early knowledge reduces unnecessary exploration during the reasoning process, enabling the model to focus more effectively on relevant information subsets. Moreover, EKA proves effective as a versatile, training-free inference strategy that scales seamlessly to large models. Generalization tests across diverse datasets and retrieval corpora confirm the robustness of our approach. Overall, EKA advances the state-of-the-art in iterative RAG systems while illuminating the critical interplay between structured reasoning and efficient exploration in reinforcement learning-augmented frameworks. The code is released at https://github.com/yxzwang/EarlyKnowledgeAlignment{Github}.

基于早期知识对齐的多跳推理

Multi-hop Reasoning via Early Knowledge Alignment

摘要

Support