PAST-Bench：个人智能体中递归自我改进基础的基准测试

摘要

递归自我改进要求智能体将积累的经验转化为更好的未来行为。个人AI智能体为研究这一能力提供了具体场景，因为它们在会话之间保留了偏好、任务历史、工具例程以及已习得的技能。然而，保留的经验是否确实能随时间提升智能体的表现，尚未经过系统测试。我们提出了PAST-Bench，一个专门用于隔离这一问题的基准。每个智能体在匹配条件下运行按序排列的新会话任务序列，并在这些条件下分别开启和关闭保留经验。该基准涵盖记忆、程序复用、信息收集和更新四类能力，共26个场景和204个回合。我们既报告后续任务的收益，也报告这些收益是否遵循预期的保存、检索和更新路径。在七个基础模型和四个智能体框架上，改进是真实存在的，但在不同能力上表现不均衡。表面增益相同的智能体，其增益是否有预期路径的证据支持可能存在显著差异。基于这些发现，我们开发了Hermes+，它在智能体循环的各阶段扩展了Hermes，加入了五项针对性干预。Hermes+提高了从保留经验中获得的平均收益，并提供了更清晰的路径证据，在需要替换过时状态的任务上改进最为显著，但效果仍取决于具体能力和模型。总体上，PAST-Bench和Hermes+为研究持久化智能体如何从保留经验发展到通过经验进行系统性改进提供了评估与诊断基础。代码：https://github.com/Gen-Verse/PAST-Bench

English

Recursive self-improvement requires agents to turn accumulated experience into better future behavior. Personal AI agents offer a concrete setting for studying this capability because they retain preferences, task histories, tool routines, and learned skills across sessions. Yet whether retained experience actually improves them over time has not been systematically tested. We introduce PAST-Bench, a benchmark designed to isolate this question. Each agent runs through ordered sequences of fresh-session tasks under matched conditions that turn retained experience on and off. It spans 26 scenarios and 204 episodes across memory, procedural reuse, information gathering, and update. We report both later-task gains and whether those gains follow the intended save, retrieve, and update pathway. Across seven base models and four agent frameworks, improvement is real but uneven across capabilities. Agents with the same headline gain can differ markedly in whether that gain is supported by evidence of the intended pathway. Guided by these findings, we develop Hermes+, which extends Hermes with five targeted interventions across stages of the agent loop. Hermes+ raises the average gain from retained experience and provides clearer pathway evidence, with its strongest improvement on tasks requiring outdated state to be replaced, although the effect remains capability- and model-dependent. Together, PAST-Bench and Hermes+ provide an evaluation and diagnostic foundation for studying how persistent agents can progress from retaining experience to systematically improving through it. Code: https://github.com/Gen-Verse/PAST-Bench