AutoWorldModel-Bench：以状态为中心的自动化世界模型研究基准

摘要

世界建模是一个尚未定型的领域：架构、训练目标与状态表示之间以复杂的方式相互作用，且没有任何单一方案能在各类环境中占据主导地位。这使得它成为检验AI编码智能体作为自主研究者的理想试验场——在这一设定中，改进方向并非预先指定，这与当前智能体基准中以按规格工程为主的任务形成鲜明对比。我们提出AutoWorldModel-Bench，一个闭环基准测试，在此基准中，前沿编码智能体在固定计算预算下自主改进所提供的世界模型起点代码。该基准涵盖八个游戏环境，并采用统一的结构化状态表示——从每个游戏中提取的真实实体状态，通过共享的张量格式进行消费——从而将动力学建模与感知相分离，使每次运行仅需数分钟即可完成迭代。在64次会话中，Codex-5.4和Claude Opus 4.6在63次中改进了起点模型；在91%的会话中，获胜的修改是非平凡的科研式改动——新的目标函数、表示方式、 rollout 过程或架构变更——而非超参数调整。我们的基准提供了一个在开放式研究中评估前沿编码智能体的场景，而非按规格工程问题。

English

World modeling is an unsettled field: architectures, training objectives, and state representations interact in complex ways, and no single recipe dominates across environments. This makes it an ideal testbed for AI coding agents acting as autonomous researchers--a setting in which the improvement direction is not specified in advance, unlike the engineering-to-spec tasks that dominate current agent benchmarks. We introduce AutoWorldModel-Bench, a closed-loop benchmark in which frontier coding agents autonomously improve a provided world-model starter under a fixed compute budget. The benchmark spans eight game environments under a unified structured-state representation--ground-truth entity state extracted from each game and consumed through a shared tensor format--which isolates dynamics modeling from perception and enables minutes-per-run iteration. Across 64 sessions, Codex-5.4 and Claude Opus 4.6 improve their starter on 63; in 91% of sessions the winning edit is a non-trivial research-style modification--a new objective, representation, rollout procedure, or architectural change--rather than a hyperparameter tweak. Our benchmark offers a setting in which frontier coding agents can be evaluated on open-ended research rather than engineering-to-spec problems.