ZimaBlue：通过可扩展视频预训练演化可泛化的世界动作模型

摘要

机器人操作面临一个根本性的扩展挑战：鲁棒的泛化需要广泛的物理经验，然而带有动作标注的机器人轨迹采集成本高昂，且多样性天然受限。第一人称视频提供了一种可扩展性远为更高的具身经验来源，能够捕获跨多样环境中的物体交互、接触动力学、工具使用以及长时程行为。核心挑战在于如何将这种丰富但无动作标注的经验转化为有效的机器人控制。我们提出ZimaBlue，一个从大规模视频中学习可泛化世界动作模型（WAMs）的可扩展框架。ZimaBlue遵循三阶段训练课程：首先在大规模人类和机器人第一人称视频上进行因果具身视频预训练，然后通过采用统一动作表示的视频-动作中间训练，将所学到的视觉动力学锚定到异构机器人轨迹上，最后将模型特化到目标机器人以进行部署。为了使生成式WAMs适用于实时控制，ZimaBlue进一步采用异步快-慢双系统架构，其中高容量的慢速世界模型提供可泛化的时空表示，而轻量级的快速分支能够在NVIDIA RTX 4090上实现30 Hz的动作预测。在真实机器人的零样本评估中，将训练数据从仅目标机器人数据扩展到超过120,000小时的具身视频，成功率从36.1%提升至77.8%。ZimaBlue在多个基准测试中进一步展现出强劲性能，在未见任务上的提升尤为显著。

English

Robotic manipulation faces a fundamental scaling challenge: robust generalization demands broad physical experience, yet action-labeled robot trajectories are expensive to collect and inherently limited in diversity. Egocentric videos offer a far more scalable source of embodied experience, capturing object interactions, contact dynamics, tool use, and long-horizon behaviors across diverse environments. The central challenge is how to convert this abundant but action-free experience into effective robot control. We introduce ZimaBlue, a scalable framework for learning generalizable World Action Models (WAMs) from large-scale video. ZimaBlue follows a three-stage training curriculum: it first performs causal embodied video pre-training on large-scale human and robot egocentric videos, then grounds the learned visual dynamics in heterogeneous robot trajectories through video-action mid-training with a unified action representation, and finally specializes the model to a target robot for deployment. To make generative WAMs practical for real-time control, ZimaBluefurther adopts an asynchronous Slow-Fast dual-system architecture, where a high-capacity Slow world model provides generalizable spatiotemporal representations and a lightweight Fast branch enables 30 Hz action prediction on NVIDIA RTX 4090. On real-robot zero-shot evaluations, scaling from target-robot data alone to over 120,000 hours of embodied video improves success from 36.1% to 77.8%. ZimaBlue further delivers strong performance across multiple benchmarks, with particularly pronounced gains on unseen tasks.