基于表征自编码器的多人交互世界模型

摘要

我们介绍了首个面向由复杂物理交互主导的高度动态环境的多智能体世界模型。单智能体世界模型将其他智能体视为环境的一部分，而我们的模型则基于多个智能体的动作流进行条件建模，学会将场景变化归因于正确的玩家，并在其任意动作组合下保持一致性。我们在《火箭联盟》这款游戏中研究这一问题，玩家在快速、紧密耦合的动力学机制下进行竞争与合作。利用公开机器人收集的一万小时游戏数据进行训练，我们拥有50亿参数的潜扩散模型能够实时生成四人对战，在单个Nvidia B200 GPU上实现每秒20帧的生成速度。尽管仅在短片段上训练，其推演稳定性远超训练范围：分布质量在长达五分钟（我们测量的最长时间范围）内保持稳定，实践中我们观察到推演持续数小时且无任何崩溃迹象。我们系统性地研究了核心设计选择：视频编解码器、生成目标以及多智能体条件方案。此外，我们刻画了行为随模型和数据规模的变化，包括涌现出的能力以及持续存在的失败模式。我们还开发了针对性评估，探测模型对物理规律的理解而不仅是视觉表现。为支持多智能体世界模型的持续研究，我们发布了数据集、完整的训练和推理代码库以及一个实时演示。

English

We introduce the first multiplayer world model for highly dynamic environments governed by complex physical interactions. Whereas single-player world models treat the other agents as part of the environment, ours conditions on the action streams of multiple agents, learning to attribute changes in the scene to the correct player and to stay coherent under arbitrary combinations of their actions. We study this problem in the game of Rocket League, where players compete and cooperate under fast, tightly coupled dynamics. Trained on 10,000 hours of gameplay collected with publicly available bots, our 5-billion-parameter latent diffusion model generates four-player matches in real time, producing 20 frames per second on a single Nvidia B200 GPU. Although trained only on short clips, its rollouts stay stable far beyond the training horizon: distributional quality holds steady out to five minutes, the longest horizon we measure, and in practice we observe rollouts continuing for hours with no sign of collapse. We systematically investigate the central design choices: the video codec, the generative objective, and the multiplayer conditioning scheme. In addition, we characterize how behavior changes with model and data scale, including the capabilities that emerge and the failure modes that persist. We further develop targeted evaluations that probe the model's physical understanding rather than visual appearance alone. To support continued research on multiplayer world models, we release our dataset, our full training and inference codebase, and a live demo.