世界引导：条件空间中的世界建模与动作生成

摘要

利用未来观测建模来促进动作生成，为增强视觉-语言-动作模型的性能开辟了前景广阔的路径。然而现有方法难以在维持高效、可预测的未来表征与保留足够细粒度信息以指导精确动作生成之间实现平衡。为解决这一局限，我们提出WoG（世界引导）框架，通过将未来观测映射为紧凑条件并注入动作推理流程。该框架训练VLA模型同步预测这些压缩条件与未来动作，从而在条件空间内实现有效的动作推理世界建模。我们证明对此条件空间的建模与预测不仅能促进细粒度动作生成，还展现出卓越的泛化能力，且能有效从海量人类操作视频中学习。在仿真与真实环境中的大量实验表明，本方法显著优于基于未来预测的现有方法。项目页面详见：https://selen-suyue.github.io/WoGNet/

English

Leveraging future observation modeling to facilitate action generation presents a promising avenue for enhancing the capabilities of Vision-Language-Action (VLA) models. However, existing approaches struggle to strike a balance between maintaining efficient, predictable future representations and preserving sufficient fine-grained information to guide precise action generation. To address this limitation, we propose WoG (World Guidance), a framework that maps future observations into compact conditions by injecting them into the action inference pipeline. The VLA is then trained to simultaneously predict these compressed conditions alongside future actions, thereby achieving effective world modeling within the condition space for action inference. We demonstrate that modeling and predicting this condition space not only facilitates fine-grained action generation but also exhibits superior generalization capabilities. Moreover, it learns effectively from substantial human manipulation videos. Extensive experiments across both simulation and real-world environments validate that our method significantly outperforms existing methods based on future prediction. Project page is available at: https://selen-suyue.github.io/WoGNet/

世界引导：条件空间中的世界建模与动作生成

World Guidance: World Modeling in Condition Space for Action Generation

摘要

Support