N_0-TWAM：面向富接触操作的触觉原生世界-动作模型的扩展

摘要

我们提出 N_0-TWAM，一种用于接触密集型操作的触觉原生世界-动作模型，可同时预测未来的视觉与未来的接触。据我们所知，这是首个大规模训练的触觉世界-动作模型，并在接触密集型任务上展现出强大能力。我们利用视觉-触觉联合训练，在覆盖六种机器人本体和 450 个任务的触觉丰富演示上对 N_0-TWAM 进行大规模预训练。我们采用 NeoForce——一种统一的基于力的触觉表示——来形成物理上有依据的接触信号，从而对动作生成进行条件约束。为改善长时程、多阶段操作，我们引入触觉接触事件来进行任务阶段划分，并在执行过程中推进这些事件。为实现实时效率，我们采用非对称的 Mixture-of-Transformers 架构，将全宽度专家用于视频预测，同时以精简专家用于下游的动作和触觉预测。对真实与仿真基准的评估验证了 N_0-TWAM 在多种接触密集型任务上的能力，并展示了数据规模扩展对精确触觉和动作预测的益处。综上，N_0-TWAM 赋予世界-动作模型以预见视觉、触觉和动作的预测能力，为开放式接触密集型任务中的细粒度操作奠定坚实基础。代码库和模型检查点将公开发布，以促进触觉赋能机器人操作的进一步研究与发展。

English

We present N_0-TWAM, a tactile-native world-action model for contact-rich manipulation that predicts both future vision and future contact. To our knowledge, it is the first tactile world-action model trained at large scale, and it shows strong capability on contact-rich tasks. We pre-train N_0-TWAM at large scale with visuo-tactile joint training over tactile-rich demonstrations spanning six embodiments and 450 tasks. We use NeoForce, a unified force-based tactile representation, to form a physically grounded contact signal that conditions action generation. To improve long-horizon and multi-stage manipulation, we introduce tactile contact events for task staging and advance through them during execution. For real-time efficiency, we adopt an asymmetric Mixture-of-Transformers architecture that pairs a full-width expert for video prediction with slim experts for downstream action and tactile prediction. Evaluations on both real and simulated benchmarks justify the capabilities of N_0-TWAM across a range of contact-rich tasks, and demonstrate the benefit of data scaling for precise tactile and action prediction. In summary, N_0-TWAM endows a world-action model with predictive capabilities to foresee vision, touch and action, building a solid foundation for fine-grained manipulation on open contact-rich tasks. The codebase and model checkpoints will be made publicly available to foster further research and development in tactile-enabled robotic manipulation.