DreamX-Phi 1.0：用于机器人操作的动作条件视频世界模型

摘要

我们提出DreamX-Phi 1.0，一个面向机器人操作的动作条件视频世界模型。给定一个观测帧、一条语言指令以及一段由末端执行器位姿和夹爪状态组成的指定动作序列，该模型能够预测后续的观测结果。然而，仅有逼真性并不能保证忠实性：一段看似合理的生成视频仍可能移动了错误的手臂或丢失了被操作物体。为确保预测结果严格遵循每条手臂的指令路径，我们通过PRoPE风格的几何编码将每条手臂的SE(3)变换注入注意力机制中，从而保持手臂的身份标识和刚体运动结构。仅靠动作控制无法完全约束场景几何或小操作物体的演化过程。为此，我们添加了一个轻量级深度分支来建模场景级几何，并利用SAM3掩码结合冻结的V-JEPA教师模型，在整个抓取过程中维持物体的一致性。我们进一步通过分布匹配蒸馏，将多步生成器蒸馏为少步学生模型，以实现高效部署。在撰写本文时，我们的方法在WorldArena 2.0挑战赛中取得了Track 1第一名和Track 2第二名的成绩。我们的模型和代码将公开发布。

English

We present DreamX-Phi 1.0, an action-conditioned video world model for robotic manipulation that, given an observed frame, a language instruction, and a prescribed action sequence comprising end-effector poses and gripper states, predicts the resulting future observations. Yet realism alone does not guarantee faithfulness: a convincing rollout can still move the wrong arm or lose the manipulated object. To ensure the prediction respects each arm's commanded path, we inject per-arm SE(3) transformations into attention via PRoPE-style geometric encoding, preserving arm identity and rigid-motion structure. Action control alone does not fully constrain scene geometry or the evolution of small manipulated objects. We therefore add a lightweight depth branch for scene-level geometry and use SAM3 masks with a frozen V-JEPA teacher to maintain object consistency throughout grasping. We further distill the multi-step generator into a few-step student via distribution-matching distillation for efficient deployment. At the time of writing, achieves first place on Track~1 and second place on Track~2 of the WorldArena~2.0 Challenge. Our model and code will be publicly available.