ZimaBlue：透過可擴展影片預訓練演化可泛化世界行動模型

摘要

機器人操作面臨一個根本性的擴展挑戰：穩健的泛化需要廣泛的物理經驗，然而帶有動作標註的機器人軌跡不僅收集成本高昂，其多樣性也本質上受限。第一人稱影片提供了一種更具可擴展性的具身經驗來源，能夠捕捉跨越多樣環境的物體互動、接觸動力學、工具使用及長時程行為。核心挑戰在於如何將這種豐富但缺乏動作標籤的經驗轉化為有效的機器人控制。我們提出 ZimaBlue，一個從大規模影片中學習可泛化世界動作模型（WAMs）的可擴展框架。ZimaBlue 遵循三階段訓練課程：首先在大規模人類與機器人第一人稱影片上進行因果具身影片預訓練；接著透過影片-動作中間訓練，以統一的動作表示將所學習的視覺動力學扎根於異質機器人軌跡中；最後將模型特化至目標機器人以進行部署。為使生成式 WAMs 能實際應用於即時控制，ZimaBlue 進一步採用了非同步慢-快雙系統架構，其中高容量的慢速世界模型提供可泛化的時空表示，而輕量的快速分支則能在 NVIDIA RTX 4090 上實現 30 Hz 的動作預測。在真實機器人零樣本評估中，將訓練資料從僅有目標機器人資料擴展至超過 120,000 小時的具身影片，使成功率從 36.1% 提升至 77.8%。ZimaBlue 在多個基準測試上均展現強勁性能，尤其在未見任務上的增益最為顯著。

English

Robotic manipulation faces a fundamental scaling challenge: robust generalization demands broad physical experience, yet action-labeled robot trajectories are expensive to collect and inherently limited in diversity. Egocentric videos offer a far more scalable source of embodied experience, capturing object interactions, contact dynamics, tool use, and long-horizon behaviors across diverse environments. The central challenge is how to convert this abundant but action-free experience into effective robot control. We introduce ZimaBlue, a scalable framework for learning generalizable World Action Models (WAMs) from large-scale video. ZimaBlue follows a three-stage training curriculum: it first performs causal embodied video pre-training on large-scale human and robot egocentric videos, then grounds the learned visual dynamics in heterogeneous robot trajectories through video-action mid-training with a unified action representation, and finally specializes the model to a target robot for deployment. To make generative WAMs practical for real-time control, ZimaBluefurther adopts an asynchronous Slow-Fast dual-system architecture, where a high-capacity Slow world model provides generalizable spatiotemporal representations and a lightweight Fast branch enables 30 Hz action prediction on NVIDIA RTX 4090. On real-robot zero-shot evaluations, scaling from target-robot data alone to over 120,000 hours of embodied video improves success from 36.1% to 77.8%. ZimaBlue further delivers strong performance across multiple benchmarks, with particularly pronounced gains on unseen tasks.