AREX：迈向递归自改进的深度研究智能体

摘要

深度研究要求智能体找到同时满足多重约束的答案。发现这类答案成本高昂，而验证候选答案时，通常可将该过程分解为可操作的逐条约束检查。这种发现-验证不对称性表明，研究智能体不应仅是延长搜索时间——它应当通过验证中间结果，并利用部分已验证的状态指导后续优化，递归地改进当前答案。我们提出AREX——一类递归自改进（RSI）深度研究智能体。AREX在内层研究循环（收集证据、构建临时答案）与外层自改进循环（逐条约束审计答案、识别未解决的断言、发起针对性后续研究）之间交替运行。为在长周期内维持RSI，AREX学习了一种自主上下文更新工具，将不断累积的交互历史压缩为紧凑的改进状态，保留已验证的证据与未解决的约束，无需依赖外部模型。我们通过智能体中间训练和长跨度强化学习，在已验证的合成任务与高质量轨迹上训练AREX。为缓解长跨度学习中稀疏最终奖励的问题，我们强化了关键步骤——即获取决定性证据或纠正错误研究方向的步骤。我们实现了密集4B模型与122B-A10B混合专家模型。在BrowseComp、WideSearch、DeepSearchQA、人类最后考试（HLE）以及其他推理与工具使用基准测试中，AREX大幅超越同规模基线模型，并与使用显著更多激活参数量的模型保持竞争力。

English

Deep research requires agents to find answers that jointly satisfy multiple constraints. Discovering such answers is costly, whereas verifying a candidate can often be decomposed into tractable constraint-wise checks. This discovery--verification asymmetry suggests that a research agent should do more than simply search longer: it should recursively improve its current answer by verifying intermediate results and using the partially verified state to guide subsequent refinement. We introduce AREX, a family of Recursively Self-Improving (RSI) deep research agents. AREX alternates between an inner research loop that gathers evidence and constructs a provisional answer, and an outer self-improvement loop that audits the answer constraint-wise, identifies unresolved claims, and launches targeted follow-up research. To sustain RSI over long horizons, AREX learns an autonomous context-update tool that compresses growing interaction history into a compact improvement state preserving verified evidence and unresolved constraints, without relying on an external model. We train AREX on verified synthetic tasks and high-quality trajectories through agentic mid-training and long-horizon reinforcement learning. To mitigate sparse final rewards during long horizon learning, we emphasize key steps where decisive evidence is acquired or erroneous research directions are corrected. We instantiate a dense 4B model and a 122B-A10B Mixture-of-Experts model. Across BrowseComp, WideSearch, DeepSearchQA, Humanity's Last Exam (HLE), and other reasoning and tool-use benchmarks, AREX substantially outperforms comparable-scale baselines and remains competitive with models using substantially more activated parameters.