PACE-Bench：动态环境中基于代码演化的物理自适应基准测试

摘要

自我进化智能体能够从交互经验中改进未来行为，然而现有评估通常仅在固定执行条件下进行优化，并未测试当这些条件发生变化后的恢复能力。为填补这一空白，我们提出PACE-Bench（基于代码演化的物理适配，Physics Adaptation via Code Evolution），这是一个基于模拟器的基准测试，涵盖六个物理域中的144个源到目标适配对。每个适配对将一个源环境与一个具有相同目标和接口的变异目标环境相连接。在源环境中可以成功运行的代码驱动设计会在目标环境中失效，智能体必须利用诊断性沙箱反馈，在有限的尝试预算内将其迭代适配为可工作的目标设计。我们比较了来自四种范式的十种自我进化方法。该基准测试远未饱和：Reflexion + Qwen3-14B仅在全部基准测试对的35.9%上取得成功，而GPT-5.5在完整预算下解决了静力学子集的66.7%。这些结果共同表明，基于模拟器的反思比未经验证的自我修正更为可靠，而记忆机制会将智能体锚定在早期设计上，宽泛的树搜索则只探索而不收敛。即使揭示确切的物理变化也无法提升性能上限，这表明核心瓶颈在于机制重设计而非参数推断。数据和代码可在https://github.com/thunlp/PACE-Bench获取。

English

Self-evolving agents improve future behavior from interaction experience, yet existing evaluations typically optimize under fixed execution conditions and do not test recovery after those conditions change. To address this gap, we introduce PACE-Bench (Physics Adaptation via Code Evolution), a simulator-grounded benchmark of 144 source-to-target adaptation pairs across six physics domains. Each pair links a source environment to a mutated target environment with the same goal and interface. A code-driven design that succeeds in the source fails in the target, where agents must iteratively adapt it into a working target design using diagnostic sandbox feedback within a limited attempt budget. We compare ten self-evolving methods from four paradigms. The benchmark remains far from saturated: Reflexion + Qwen3-14B succeeds on only 35.9\% of full-benchmark pairs, while GPT-5.5 solves 66.7\% of the Statics subset under the full budget. Together, these results show that simulator-grounded reflection is more reliable than unverified self-revision, while memory anchors agents to early designs and broad tree search explores without converging. Even revealing exact physical changes does not raise the performance ceiling, pointing to mechanism redesign rather than parameter inference as the central bottleneck. Data and code are available at https://github.com/thunlp/PACE-Bench.