DynamicRAG：利用大型語言模型的輸出作為反饋，實現檢索增強生成中的動態重排序

DynamicRAG: Leveraging Outputs of Large Language Model as Feedback for Dynamic Reranking in Retrieval-Augmented Generation

May 12, 2025

作者: Jiashuo Sun, Xianrui Zhong, Sizhe Zhou, Jiawei Han

cs.AI

摘要

檢索增強生成（RAG）系統結合了大型語言模型（LLMs）與外部知識檢索，使其在知識密集型任務中表現卓越。這些系統中一個關鍵但常被忽視的組件是重排序器，它精煉檢索到的文件以提升生成質量與可解釋性。選擇最佳文件數量（k）的挑戰仍未解決：過少可能遺漏關鍵信息，過多則引入噪音與低效。儘管近期研究探索了基於LLM的重排序器，它們主要依賴模型內部知識，並忽略了LLM能提供的豐富監督信號，例如利用回應質量作為優化重排序決策的反饋。本文提出DynamicRAG，一種新穎的RAG框架，其中重排序器根據查詢動態調整檢索文件的順序與數量。我們將重排序器建模為通過強化學習（RL）優化的代理，利用源自LLM輸出質量的獎勵。在七個知識密集型數據集上，DynamicRAG展現了卓越性能，達到了業界領先水平。模型、數據與代碼可在https://github.com/GasolSun36/DynamicRAG 獲取。

English

Retrieval-augmented generation (RAG) systems combine large language models (LLMs) with external knowledge retrieval, making them highly effective for knowledge-intensive tasks. A crucial but often under-explored component of these systems is the reranker, which refines retrieved documents to enhance generation quality and explainability. The challenge of selecting the optimal number of documents (k) remains unsolved: too few may omit critical information, while too many introduce noise and inefficiencies. Although recent studies have explored LLM-based rerankers, they primarily leverage internal model knowledge and overlook the rich supervisory signals that LLMs can provide, such as using response quality as feedback for optimizing reranking decisions. In this paper, we propose DynamicRAG, a novel RAG framework where the reranker dynamically adjusts both the order and number of retrieved documents based on the query. We model the reranker as an agent optimized through reinforcement learning (RL), using rewards derived from LLM output quality. Across seven knowledge-intensive datasets, DynamicRAG demonstrates superior performance, achieving state-of-the-art results. The model, data and code are available at https://github.com/GasolSun36/DynamicRAG