MiniWorld：从零开始普及视频世界模型训练

摘要

视频世界模型以历史观测和控制信号为条件预测未来观测，通过自回归状态转换实现长程生成。与主要捕捉视觉外观和运动的传统视频生成模型不同，视频世界模型学习在智能体动作作用下支配环境演化的底层动力学，为具身智能和交互式仿真奠定基础。近期进展主要依赖于通过后训练或蒸馏来适配预训练的视频生成模型。尽管有效，这些方法通常需要复杂的训练流程、大量的计算资源，并且存在双向预训练与因果流式推理之间的不匹配问题。近期研究表明，从头训练自回归视频世界模型是可行且可扩展的。然而，社区仍然缺乏一个轻量级、透明且完全可复现的基线，能够在适度的计算资源下端到端训练。我们提出了MiniWorld，一个从头训练流式视频世界模型的可复现框架。MiniWorld采用块因果视频扩散Transformer，在预训练视频VAE的潜空间中使用流匹配进行训练。基于Diffusion Forcing，它采用分块非递减噪声调度和两阶段持续训练来改善时序建模和稳定性。在推理过程中，MiniWorld将滚动KV缓存与流水线异步去噪相结合，在有界计算下实现高效的流式生成。整个模型可以在单个8-GPU服务器上数天内完成训练。通过发布训练与推理代码库以及预训练检查点，我们希望MiniWorld能够促进视频世界建模的未来研究。

English

Video world models predict future observations conditioned on historical observations and control signals, enabling long-horizon generation through autoregressive state transitions. Unlike conventional video generation models that primarily capture visual appearance and motion, video world models learn the underlying dynamics governing environment evolution under agent actions, providing a foundation for embodied AI and interactive simulation. Recent progress has largely relied on adapting pretrained video generation models through post-training or distillation. Although effective, these approaches often require complex training pipelines, substantial computational resources, and suffer from the mismatch between bidirectional pretraining and causal streaming inference. Recent studies have shown that training autoregressive video world models from scratch is feasible and scalable. However, the community still lacks a lightweight, transparent, and fully reproducible baseline trainable end-to-end with modest computational resources. We present MiniWorld, a reproducible framework for training streaming video world models from scratch. MiniWorld employs a block-causal Video Diffusion Transformer trained with Flow Matching in the latent space of a pretrained Video VAE. Building on Diffusion Forcing, it adopts a chunk-wise non-decreasing noise schedule and two-stage continued training to improve temporal modeling and stability. During inference, MiniWorld combines a rolling KV cache with pipelined asynchronous denoising for efficient streaming generation under bounded computation. The entire model can be trained within several days on a single 8-GPU server. By releasing the training and inference codebase and pretrained checkpoints, we hope MiniWorld will facilitate future research on video world modeling.