智能體驅動的遊戲開發：擴展世界模型的可驗證軌跡數據引擎

摘要

扩展世界模型的一种常见策略是利用更多算力在更多抓取的视频上进行训练。我们认为这一策略效率低下：扩展世界模型还需要一个能够提供有根基的奖励信号的递归数据引擎。代码智能体的成功恰好说明了这一点为何至关重要。由于代码是可执行的，编译器和运行时可以为大语言模型（LLM）的强化学习（RL）后训练提供高质量的奖励。相比之下，空间生成仍主要依赖诸如CLIP分数之类的模糊代理指标。这些信号既模糊又带有偏差，难以支撑强化学习后训练。与此相比，游戏开发为空间世界模型提供了一个缺失的奖励环境。由游戏引擎编码的场景是一种可执行的世界规范：引擎能够高效地检查碰撞、物理、可导航性以及有界的可玩性，而开发者则通过判断场景是否应当被接受来提供全局验证信号。游戏开发还为强化学习后训练提供了真实世界中的长时程轨迹数据。因此，我们提出一种基于人类与引擎联合验证的强化学习范式（RLHEV），这是一种后训练范式，它将稠密的引擎信号与来自开发过程的隐式人类接受反馈相结合。

English

A common strategy for scaling world models is to train on more crawled video with more compute. We argue that this strategy is inefficient: scaling world models also requires a recursive data engine that offers grounded reward signals. The success of code agents illustrates why this matters. As code is executable, compilers and runtimes can provide high-quality rewards for Reinforcement Learning (RL) post-training of LLMs. By contrast, spatial generation still relies largely on fuzzy proxies such as CLIP scores. These signals are fuzzy and biased, making them hard to support RL post-training. Compared with these, game development provides a missing reward environment for spatial world models. A scene encoded by a game engine is an executable world specification: the engine can efficiently check collision, physics, navigability and bounded playability, while the developer provides the global verification signal by judging whether the scene should be accepted. Game development also provides real-world long-horizon trajectory data for RL post-training. We therefore propose Reinforcement Learning with Human-Engine Verification (RLHEV), a post-training paradigm that combines dense engine signals with implicit human acceptance feedback from the development process.

智能體驅動的遊戲開發：擴展世界模型的可驗證軌跡數據引擎

Agentic Game Development as a Verifiable Trajectory Data Engine for Scaling World Models

摘要

Support