GRASP：面向代理式RAG的粒度感知搜尋策略

摘要

代理式檢索增強生成（RAG）透過允許語言模型反覆進行推理、生成搜尋查詢、檢索證據及預測答案，從而擴展了靜態RAG。然而，模型在決定何時檢索、是否使用詞彙匹配或語義相似度，以及如何控制上下文粒度以避免無關詞彙干擾代理推理方面，仍面臨挑戰。本文提出GRASP，一個用於訓練代理在進行多步推理時能自適應協調互補檢索工具的強化學習（RL）框架。GRASP賦予代理語義搜尋、關鍵字搜尋及段落閱讀等行動，使其能檢索句子級證據，並僅在必要時擴展更多上下文。我們透過一項同時考慮答案準確度、實證閱讀、互補搜尋及輪次效率的獎勵函數來訓練策略。在多跳推理基準上的實驗顯示，與單步檢索、基於提示的代理式RAG以及基於RL的檢索基準相比，GRASP在檢索召回率與下游問答表現上均有提升。定性分析與消融分析表明，學習到的策略發展出可解釋的略讀與掃讀行為：使用語義搜尋進行廣泛探索，利用段落閱讀進行局部驗證，並以關鍵字搜尋尋找實體特定證據。這些結果顯示，學習協調檢索信號與上下文粒度對於代理的正確推理至關重要。

English

Agentic retrieval-augmented generation (RAG) extends static RAG by allowing language models to iteratively reason, generate search queries, retrieve evidence, and predict answers. However, it remains challenging for models to decide when to retrieve, whether to use lexical matching or semantic similarity, and how to control context granularity to prevent irrelevant tokens from interfering with agent reasoning. In this paper, we introduce GRASP, a reinforcement learning (RL) framework for training agents to adaptively coordinate complementary retrieval tools during multi-step reasoning. GRASP provides the agent with semantic search, keyword search, and paragraph-reading actions, enabling it to retrieve sentence-level evidence and expand further context only when needed. We train the policy with a reward that jointly accounts for answer accuracy, grounded reading, complementary search, and turn efficiency. Experiments on multi-hop reasoning benchmarks show that GRASP improves both retrieval recall and downstream question answering performance compared with single-step retrieval, prompting-based agentic RAG, and RL-based retrieval baselines. Qualitative and ablation analyses show that the learned policy develops interpretable skimming and scanning behavior: it uses semantic search for broad exploration, paragraph reading for local verification, and keyword search for entity-specific evidence. These results suggest that learning to coordinate retrieval signals and context granularity is critical for agent's correct reasoning.