智能体式游戏开发：用于扩展世界模型的可验证轨迹数据引擎

摘要

扩展世界模型的一种常见策略是投入更多算力在更多抓取视频上进行训练。我们认为这一策略效率低下：扩展世界模型还需要一个能够提供有依据奖励信号的递归数据引擎。代码智能体的成功恰好说明了这一点为何重要。由于代码是可执行的，编译器和运行时可以为大语言模型的强化学习（RL）后训练提供高质量的奖励。相比之下，空间生成仍然在很大程度上依赖诸如CLIP分数之类的模糊代理指标。这些信号模糊且有偏，难以支撑强化学习后训练。与此相比，游戏开发为空间世界模型提供了一个缺失的奖励环境。由游戏引擎编码的场景是一种可执行的世界规范：引擎能够高效地检查碰撞、物理、可导航性和有界可玩性，而开发者则通过判断场景是否应当被接受来提供全局验证信号。游戏开发还提供了真实世界中的长视界轨迹数据，可用于强化学习后训练。因此，我们提出基于人与引擎联合验证的强化学习（RLHEV），这是一种结合了密集引擎信号与开发过程中隐式人工接受反馈的后训练范式。

English

A common strategy for scaling world models is to train on more crawled video with more compute. We argue that this strategy is inefficient: scaling world models also requires a recursive data engine that offers grounded reward signals. The success of code agents illustrates why this matters. As code is executable, compilers and runtimes can provide high-quality rewards for Reinforcement Learning (RL) post-training of LLMs. By contrast, spatial generation still relies largely on fuzzy proxies such as CLIP scores. These signals are fuzzy and biased, making them hard to support RL post-training. Compared with these, game development provides a missing reward environment for spatial world models. A scene encoded by a game engine is an executable world specification: the engine can efficiently check collision, physics, navigability and bounded playability, while the developer provides the global verification signal by judging whether the scene should be accepted. Game development also provides real-world long-horizon trajectory data for RL post-training. We therefore propose Reinforcement Learning with Human-Engine Verification (RLHEV), a post-training paradigm that combines dense engine signals with implicit human acceptance feedback from the development process.

智能体式游戏开发：用于扩展世界模型的可验证轨迹数据引擎

Agentic Game Development as a Verifiable Trajectory Data Engine for Scaling World Models

摘要

Support