ABSeeker：通过答案回溯信用分配训练长程搜索智能体

摘要

长程搜索智能体必须执行多个连续动作（步骤）来搜索、检索、验证并整合证据，最终得出答案。然而，现有的智能体训练方法通常在监督微调（SFT）和强化学习（RL）过程中对轨迹中的所有步骤一视同仁，未能区分有用动作与错误或冗余动作。本文提出答案回溯信用分配（Answer-Backtracked Credit Assignment，ABC），一种用于训练长程搜索智能体的细粒度信用分配框架，通过将稀疏的轨迹级结果转化为密集的步级监督信号，奖励有用动作（即使在失败的轨迹中），同时抑制错误或冗余动作。具体而言，给定一个可能晦涩的查询及其对应的标准答案，ABC首先执行答案回溯线索恢复（Answer-Backtracked Clue Recovery），从答案出发反向追踪，恢复解答问题所需的中间线索；随后应用线索锚定步骤评分（Clue-Anchored Step Scoring），对照这些线索评估每个搜索步骤，将稀疏的二元结果监督转化为密集的步级奖励。基于这些奖励，我们开发了ABC-SFT（对每轮损失进行重新加权）和ABC-GRPO（在GRPO中使用步级分数作为奖励）。在此框架基础上，我们仅使用8.5k条示例，基于Qwen3.5-4B训练了ABSeeker。ABSeeker在BrowseComp上达到37.3%，在BrowseComp-ZH上达到39.1%。加入上下文管理后，得分进一步提升至55.3%和52.9%，显著优于同规模（4B）智能体，甚至可与更大规模（约30B）的模型相媲美。这些结果证明了答案回溯式步级信用分配在训练长程搜索智能体方面的有效性。

English

Long-horizon search agents must make multiple sequential actions (steps) to search, retrieve, verify, and integrate evidence to reach a final answer. However, existing methods for training these agents typically treat all steps within a trajectory uniformly during both supervised fine-tuning (SFT) and reinforcement learning (RL), failing to distinguish useful actions from erroneous or redundant ones. In this paper, we propose Answer-Backtracked Credit Assignment (ABC), a fine-grained credit assignment framework for training long-horizon search agents by converting sparse trajectory-level outcomes into dense step-level supervision that rewards useful actions (even in failed trajectories) while suppressing erroneous or redundant actions. Specifically, given a potentially obscure query and its corresponding ground-truth answer, ABC first performs Answer-Backtracked Clue Recovery, which traces back from the answer to recover intermediate clues required to solve the question. It then applies Clue-Anchored Step Scoring to evaluate each search step against these clues, converting sparse binary outcome supervision into dense step-level rewards. Based on these rewards, we develop ABC-SFT, which reweights the loss of each turn, and ABC-GRPO, which uses the step-level scores as rewards in GRPO. Building on this framework, we train ABSeeker based on Qwen3.5-4B with only 8.5k examples. ABSeeker achieves 37.3% on BrowseComp and 39.1% on BrowseComp-ZH. With context management, the scores further improve to 55.3% and 52.9%, respectively, significantly outperforming same-scale (4B) agents and even matching the performance of larger ones (approximately 30B). These results demonstrate the effectiveness of answer-backtracked step-level credit assignment for training long-horizon search agents.