智能搜索：基于奖励引导的查询优化处理在搜索代理中的应用

SmartSearch: Process Reward-Guided Query Refinement for Search Agents

January 8, 2026

作者: Tongyu Wen, Guanting Dong, Zhicheng Dou

cs.AI

摘要

基于大语言模型（LLM）的搜索代理已被证明能通过整合信息检索能力有效解决知识密集型问题。现有研究主要聚焦于优化搜索代理的推理范式，却忽视了推理过程中中间检索查询的质量问题。这导致生成的查询往往存在偏差，引发意外检索结果，最终制约搜索代理的整体效能。为解决该问题，我们提出SmartSearch框架，其核心包含两项机制：（1）过程奖励机制：通过双层级信用评估对每个中间检索查询质量实施细粒度监督；（2）查询优化机制：通过选择性优化低质量检索查询，并基于优化结果重新生成后续搜索轮次，提升查询生成质量。为使搜索代理在过程奖励引导下逐步内化查询质量提升能力，我们设计了三阶段课程学习框架，引导代理经历从模仿、对齐到泛化的渐进过程。实验结果表明，SmartSearch在各项基准测试中均优于现有基线方法，定量分析进一步验证其在搜索效率与查询质量方面的显著提升。代码已开源：https://github.com/MYVAE/SmartSearch。

English

Large language model (LLM)-based search agents have proven promising for addressing knowledge-intensive problems by incorporating information retrieval capabilities. Existing works largely focus on optimizing the reasoning paradigms of search agents, yet the quality of intermediate search queries during reasoning remains overlooked. As a result, the generated queries often remain inaccurate, leading to unexpected retrieval results and ultimately limiting search agents' overall effectiveness. To mitigate this issue, we introduce SmartSearch, a framework built upon two key mechanisms: (1) Process rewards, which provide fine-grained supervision for the quality of each intermediate search query through Dual-Level Credit Assessment. (2) Query refinement, which promotes the optimization of query generation by selectively refining low-quality search queries and regenerating subsequent search rounds based on these refinements. To enable the search agent to progressively internalize the ability to improve query quality under the guidance of process rewards, we design a three-stage curriculum learning framework. This framework guides the agent through a progression from imitation, to alignment, and ultimately to generalization. Experimental results show that SmartSearch consistently surpasses existing baselines, and additional quantitative analyses further confirm its significant gains in both search efficiency and query quality. The code is available at https://github.com/MYVAE/SmartSearch.