零博士：无需训练数据的自我进化搜索代理

Dr. Zero: Self-Evolving Search Agents without Training Data

January 11, 2026

作者: Zhenrui Yue, Kartikeya Upasani, Xianjun Yang, Suyu Ge, Shaoliang Nie, Yuning Mao, Zhe Liu, Dong Wang

cs.AI

摘要

随着高质量数据获取日益困难，无数据自进化已成为一种前景广阔的范式。该方法使大语言模型能够自主生成并解决复杂问题，从而提升其推理能力。然而在多轮搜索智能体中，由于问题多样性有限且多步推理与工具调用需要大量计算资源，无数据自进化面临挑战。本研究提出Dr. Zero框架，使搜索智能体能在无训练数据条件下实现有效自进化。我们特别设计了自进化反馈循环：提议者生成多样化问题来训练基于同一基础模型初始化的求解器，随着求解器进化，它会激励提议者产出难度递增且可解决的任务，从而建立自动化课程以同步优化双智能体。为提升训练效率，我们还提出跳步分组相对策略优化（HRPO）。该方法通过聚类结构相似的问题构建组级基线，有效降低了评估单个查询难度与可解性时的采样开销。因此HRPO能在保持性能与稳定性的同时，显著减少求解器训练的计算需求。大量实验结果表明，无数据的Dr. Zero达到或超越了全监督搜索智能体的性能，证明了复杂推理与搜索能力可仅通过自进化机制涌现。

English

As high-quality data becomes increasingly difficult to obtain, data-free self-evolution has emerged as a promising paradigm. This approach allows large language models (LLMs) to autonomously generate and solve complex problems, thereby improving their reasoning capabilities. However, multi-turn search agents struggle in data-free self-evolution due to the limited question diversity and the substantial compute required for multi-step reasoning and tool using. In this work, we introduce Dr. Zero, a framework enabling search agents to effectively self-evolve without any training data. In particular, we design a self-evolution feedback loop where a proposer generates diverse questions to train a solver initialized from the same base model. As the solver evolves, it incentivizes the proposer to produce increasingly difficult yet solvable tasks, thus establishing an automated curriculum to refine both agents. To enhance training efficiency, we also introduce hop-grouped relative policy optimization (HRPO). This method clusters structurally similar questions to construct group-level baselines, effectively minimizing the sampling overhead in evaluating each query's individual difficulty and solvability. Consequently, HRPO significantly reduces the compute requirements for solver training without compromising performance or stability. Extensive experiment results demonstrate that the data-free Dr. Zero matches or surpasses fully supervised search agents, proving that complex reasoning and search capabilities can emerge solely through self-evolution.