GRASP：粒度感知的智能体检索增强生成搜索策略

摘要

智能体检索增强生成（RAG）通过允许语言模型迭代推理、生成搜索查询、检索证据并预测答案，扩展了静态RAG。然而，模型仍难以决定何时进行检索、是使用词汇匹配还是语义相似度，以及如何控制上下文粒度以防止无关标记干扰智能体推理。本文提出GRASP，一种用于训练智能体在多步推理过程中自适应协调互补检索工具的强化学习（RL）框架。GRASP为智能体提供语义搜索、关键词搜索和段落阅读三种动作，使其能够检索句子级证据，并仅在必要时扩展更多上下文。我们通过联合考虑答案准确性、基于证据的阅读、互补搜索和轮次效率的奖励函数来训练策略。在多跳推理基准上的实验表明，与单步检索、基于提示的智能体RAG以及基于强化学习的检索基线相比，GRASP在检索召回率和下游问答性能上均有提升。定性和消融分析显示，学习到的策略展现出可解释的速读与精读行为：利用语义搜索进行广泛探索，通过段落阅读进行局部验证，并借助关键词搜索获取实体特定证据。这些结果表明，学习协调检索信号和上下文粒度对于智能体的正确推理至关重要。

English

Agentic retrieval-augmented generation (RAG) extends static RAG by allowing language models to iteratively reason, generate search queries, retrieve evidence, and predict answers. However, it remains challenging for models to decide when to retrieve, whether to use lexical matching or semantic similarity, and how to control context granularity to prevent irrelevant tokens from interfering with agent reasoning. In this paper, we introduce GRASP, a reinforcement learning (RL) framework for training agents to adaptively coordinate complementary retrieval tools during multi-step reasoning. GRASP provides the agent with semantic search, keyword search, and paragraph-reading actions, enabling it to retrieve sentence-level evidence and expand further context only when needed. We train the policy with a reward that jointly accounts for answer accuracy, grounded reading, complementary search, and turn efficiency. Experiments on multi-hop reasoning benchmarks show that GRASP improves both retrieval recall and downstream question answering performance compared with single-step retrieval, prompting-based agentic RAG, and RL-based retrieval baselines. Qualitative and ablation analyses show that the learned policy develops interpretable skimming and scanning behavior: it uses semantic search for broad exploration, paragraph reading for local verification, and keyword search for entity-specific evidence. These results suggest that learning to coordinate retrieval signals and context granularity is critical for agent's correct reasoning.