PACE-Bench：動態環境中程式碼演化驅動的物理適應基準測試

摘要

自我演化代理能從互動經驗中改善未來行為，然而現有評估通常在固定執行條件下進行優化，且未測試條件改變後的恢復能力。為填補此缺口，我們提出 PACE-Bench（透過程式碼演化進行物理適應，Physics Adaptation via Code Evolution），一個以模擬器為基礎的基準測試，涵蓋六個物理領域中的144組源環境到目標環境適應任務對。每組任務對將源環境與具有相同目標和介面的變異目標環境相連結。在源環境中成功的程式碼驅動設計於目標環境中會失敗，代理必須在有限的嘗試預算內，利用診斷式沙盒回饋，迭代地將其改編為可行的目標設計。我們比較了來自四種典範的十種自我演化方法。該基準測試遠未飽和：Reflexion + Qwen3-14B 在完整基準測試中僅成功完成35.9%的任務對，而 GPT-5.5 在完整預算下僅解決了 Statics 子集的66.7%。綜合這些結果顯示，以模擬器為基礎的反思比未經驗證的自我修訂更可靠，而記憶會使代理錨定於早期設計，廣泛的樹狀搜尋則探索卻無法收斂。即使揭露確切的物理變化也未能提升效能上限，這表明機制重新設計而非參數推斷才是核心瓶頸。資料與程式碼可於 https://github.com/thunlp/PACE-Bench 取得。

English

Self-evolving agents improve future behavior from interaction experience, yet existing evaluations typically optimize under fixed execution conditions and do not test recovery after those conditions change. To address this gap, we introduce PACE-Bench (Physics Adaptation via Code Evolution), a simulator-grounded benchmark of 144 source-to-target adaptation pairs across six physics domains. Each pair links a source environment to a mutated target environment with the same goal and interface. A code-driven design that succeeds in the source fails in the target, where agents must iteratively adapt it into a working target design using diagnostic sandbox feedback within a limited attempt budget. We compare ten self-evolving methods from four paradigms. The benchmark remains far from saturated: Reflexion + Qwen3-14B succeeds on only 35.9\% of full-benchmark pairs, while GPT-5.5 solves 66.7\% of the Statics subset under the full budget. Together, these results show that simulator-grounded reflection is more reliable than unverified self-revision, while memory anchors agents to early designs and broad tree search explores without converging. Even revealing exact physical changes does not raise the performance ceiling, pointing to mechanism redesign rather than parameter inference as the central bottleneck. Data and code are available at https://github.com/thunlp/PACE-Bench.