GDPevo：評估智能體在真實商業任務上的自我進化

摘要

代理自我進化會根據先前經驗更新代理的持久狀態，並重用該狀態以更有效地解決相關任務。評估自我進化十分困難：現有基準測試對具有經濟價值的任務領域涵蓋有限，且並非總能設計訓練與測試任務，使測試階段的效能提升可歸因於訓練經驗，同時仍易受數據污染影響。我們提出了GDPevo——一個以GDP相關企業工作流程為基礎、立足於進化機制的基準測試，並配備了可全自動生成該基準的數據管線。其核心機制「規則雜交」將每個企業工作流程分解為原子業務規則，將這些規則的子集分配至訓練任務中，並在留出的測試任務中重新組合，從而使測試階段的效能提升具有可歸因性。GDPevo涵蓋客戶關係管理、企業資源規劃、金融、醫療保健、法律及數據中心工作流程。其V1版本包含12個組別共120項任務，每組各有五項訓練任務與五項留出測試任務。全自動化使該管線能在兩天內將套件擴展至24個組別共240項任務（V2版本），為數據污染問題提供了務實的應對方案。我們使用GDPevo評估了四種代理（各包含一個驅動框架與一個模型），涵蓋四種監督類型。自我進化持續提升留出測試的準確率，最高可提升16.44個百分點。然而，最佳進化代理仍遠低於完全知情的神諭上限91.6%，這表明當前代理的自我進化能力尚未充分發揮。我們已在https://github.com/Prism-Shadow/GDPevo公開釋出管線、基準測試及完整的評估結果。

English

Agent self-evolution updates an agent's persistent state from prior experience and reuses it to solve related tasks more effectively. Evaluating self-evolution is difficult: existing benchmarks provide limited coverage of economically valuable task domains, do not always design training and test tasks such that test-time gains can be attributed to training experience, and remain vulnerable to data contamination. We present GDPevo, an evolution-native benchmark grounded in GDP-related enterprise workflows, together with the fully automated data pipeline that generates it. Its core mechanism, rule hybridization, decomposes each enterprise workflow into atomic business rules, distributes subsets of these rules across training tasks, and recombines them in held-out test tasks so that test-time gains are attributable. GDPevo spans CRM, ERP, finance, healthcare, legal, and data-centric workflows. Its V1 release contains 120 tasks in 12 groups, with five training and five held-out test tasks per group. Full automation enables the pipeline to expand the suite to 240 tasks in 24 groups (V2) within two days, providing a practical response to contamination. Using GDPevo, we evaluate four agents, each comprising a harness and a model, under four supervision types. Self-evolution consistently improves held-out accuracy by up to 16.44 percentage points. But the best evolved agents remain far below the fully informed oracle ceiling of 91.6%, indicating that the self-evolution ability of current agents remains far from fully realized. We publicly release the pipeline, benchmark, and full evaluation results at https://github.com/Prism-Shadow/GDPevo.