ABSeeker：通過答案回溯的信用分配訓練長時程搜索智能體

摘要

長程搜尋代理必須執行多次依序的行動（步驟），以搜尋、檢索、驗證並整合證據，最終得出答案。然而，現有訓練這些代理的方法，無論是在監督式微調（SFT）還是強化學習（RL）中，通常都將軌跡內的所有步驟一視同仁，無法區分有用的行動與錯誤或冗餘的行動。在本文中，我們提出「答案回溯信用分配」（Answer-Backtracked Credit Assignment, ABC），這是一個細粒度的信用分配框架，用於訓練長程搜尋代理；其做法是將稀疏的軌跡層級結果轉換為密集的步驟層級監督，獎勵有用的行動（即使在失敗的軌跡中），同時抑制錯誤或冗餘的行動。具體而言，給定一個可能隱晦的查詢及其對應的標準答案，ABC 首先執行「答案回溯線索還原」，從答案回溯，還原解決該問題所需的中間線索；接著套用「線索錨定步驟評分」，根據這些線索評估每個搜尋步驟，將稀疏的二元結果監督轉換為密集的步驟層級獎勵。基於這些獎勵，我們發展了 ABC-SFT，重新加權每一回合的損失；以及 ABC-GRPO，在 GRPO 中使用步驟層級分數作為獎勵。在此框架基礎上，我們僅用 8.5k 個範例，基於 Qwen3.5-4B 訓練了 ABSeeker。ABSeeker 在 BrowseComp 上達到 37.3%，在 BrowseComp-ZH 上達到 39.1%。在加入上下文管理後，分數分別進一步提升至 55.3% 與 52.9%，顯著優於相同規模（4B）的代理，甚至能匹配較大規模（約 30B）代理的表現。這些結果證明了答案回溯的步驟層級信用分配在訓練長程搜尋代理方面的有效性。

English

Long-horizon search agents must make multiple sequential actions (steps) to search, retrieve, verify, and integrate evidence to reach a final answer. However, existing methods for training these agents typically treat all steps within a trajectory uniformly during both supervised fine-tuning (SFT) and reinforcement learning (RL), failing to distinguish useful actions from erroneous or redundant ones. In this paper, we propose Answer-Backtracked Credit Assignment (ABC), a fine-grained credit assignment framework for training long-horizon search agents by converting sparse trajectory-level outcomes into dense step-level supervision that rewards useful actions (even in failed trajectories) while suppressing erroneous or redundant actions. Specifically, given a potentially obscure query and its corresponding ground-truth answer, ABC first performs Answer-Backtracked Clue Recovery, which traces back from the answer to recover intermediate clues required to solve the question. It then applies Clue-Anchored Step Scoring to evaluate each search step against these clues, converting sparse binary outcome supervision into dense step-level rewards. Based on these rewards, we develop ABC-SFT, which reweights the loss of each turn, and ABC-GRPO, which uses the step-level scores as rewards in GRPO. Building on this framework, we train ABSeeker based on Qwen3.5-4B with only 8.5k examples. ABSeeker achieves 37.3% on BrowseComp and 39.1% on BrowseComp-ZH. With context management, the scores further improve to 55.3% and 52.9%, respectively, significantly outperforming same-scale (4B) agents and even matching the performance of larger ones (approximately 30B). These results demonstrate the effectiveness of answer-backtracked step-level credit assignment for training long-horizon search agents.