GDPevo：评估智能体在真实业务任务中的自我进化

摘要

智能体自我进化利用先前经验更新智能体的持久状态，并复用它以更有效地解决相关任务。评估自我进化十分困难：现有基准对具有经济价值的任务领域覆盖有限，且并不总能将训练任务和测试任务设计为能使测试阶段性能提升归因于训练经验，并且仍然容易受到数据污染的影响。我们提出了 GDPevo——一个原生支持自我进化的基准，以 GDP 相关的企业工作流为基础，并配套了一条全自动数据流水线来生成该基准。其核心机制——规则杂交（rule hybridization）——将每个企业工作流分解为原子业务规则，将这些规则的子集分配到训练任务中，并在留出的测试任务中重新组合这些规则，从而使测试阶段的性能提升可归因于训练经验。GDPevo 涵盖客户关系管理（CRM）、企业资源计划（ERP）、金融、医疗保健、法律以及以数据为中心的工作流。其 V1 版本包含 12 组共 120 个任务，每组有 5 个训练任务和 5 个留出测试任务。完全自动化使该流水线能够在两天内将测试套件扩展到 24 组共 240 个任务（V2），为数据污染问题提供了切实的应对方案。基于 GDPevo，我们在四种监督类型下评估了四个智能体，每个智能体由一个运行框架（harness）和一个模型组成。自我进化持续提升了留出集准确率，最高达 16.44 个百分点。但表现最好的进化后智能体仍远低于全知预言机（oracle）91.6% 的上限，这表明当前智能体的自我进化能力仍远未得到充分发挥。我们在 https://github.com/Prism-Shadow/GDPevo 公开发布了该流水线、基准以及完整的评估结果。

English

Agent self-evolution updates an agent's persistent state from prior experience and reuses it to solve related tasks more effectively. Evaluating self-evolution is difficult: existing benchmarks provide limited coverage of economically valuable task domains, do not always design training and test tasks such that test-time gains can be attributed to training experience, and remain vulnerable to data contamination. We present GDPevo, an evolution-native benchmark grounded in GDP-related enterprise workflows, together with the fully automated data pipeline that generates it. Its core mechanism, rule hybridization, decomposes each enterprise workflow into atomic business rules, distributes subsets of these rules across training tasks, and recombines them in held-out test tasks so that test-time gains are attributable. GDPevo spans CRM, ERP, finance, healthcare, legal, and data-centric workflows. Its V1 release contains 120 tasks in 12 groups, with five training and five held-out test tasks per group. Full automation enables the pipeline to expand the suite to 240 tasks in 24 groups (V2) within two days, providing a practical response to contamination. Using GDPevo, we evaluate four agents, each comprising a harness and a model, under four supervision types. Self-evolution consistently improves held-out accuracy by up to 16.44 percentage points. But the best evolved agents remain far below the fully informed oracle ceiling of 91.6%, indicating that the self-evolution ability of current agents remains far from fully realized. We publicly release the pipeline, benchmark, and full evaluation results at https://github.com/Prism-Shadow/GDPevo.