N_0-TWAM：擴展觸覺原生世界動作模型以實現高接觸操控

摘要

我們提出 N_0-TWAM，一個以觸覺為本的世界-動作模型，專用於接觸密集操作，並能同時預測未來的視覺與未來的接觸。據我們所知，這是首個大規模訓練的觸覺世界-動作模型，且在接觸密集任務上展現出強大能力。我們以視覺-觸覺聯合訓練方式，在涵蓋六種具身與 450 個任務的觸覺密集示範資料上，大規模預訓練 N_0-TWAM。我們使用 NeoForce——一種統一的基於力之觸覺表徵——來形成具有物理基礎的接觸信號，以條件化動作生成。為改善長時程與多階段操作，我們引入觸覺接觸事件以進行任務階段化，並在執行過程中依序推進這些階段。為達成即時效率，我們採用非對稱的 Mixture-of-Transformers 架構，將全寬專家用於視訊預測，並搭配精簡專家進行下游動作與觸覺預測。在真實與模擬基準上的評估，驗證了 N_0-TWAM 在各種接觸密集任務上的能力，並展現了資料規模擴展對於精確觸覺與動作預測的效益。總結而言，N_0-TWAM 賦予世界-動作模型預見視覺、觸覺與動作的預測能力，為開放式接觸密集任務上的精細操作奠定了堅實基礎。程式碼庫與模型檢查點將公開提供，以促進觸覺賦能機器人操作的後續研究與開發。

English

We present N_0-TWAM, a tactile-native world-action model for contact-rich manipulation that predicts both future vision and future contact. To our knowledge, it is the first tactile world-action model trained at large scale, and it shows strong capability on contact-rich tasks. We pre-train N_0-TWAM at large scale with visuo-tactile joint training over tactile-rich demonstrations spanning six embodiments and 450 tasks. We use NeoForce, a unified force-based tactile representation, to form a physically grounded contact signal that conditions action generation. To improve long-horizon and multi-stage manipulation, we introduce tactile contact events for task staging and advance through them during execution. For real-time efficiency, we adopt an asymmetric Mixture-of-Transformers architecture that pairs a full-width expert for video prediction with slim experts for downstream action and tactile prediction. Evaluations on both real and simulated benchmarks justify the capabilities of N_0-TWAM across a range of contact-rich tasks, and demonstrate the benefit of data scaling for precise tactile and action prediction. In summary, N_0-TWAM endows a world-action model with predictive capabilities to foresee vision, touch and action, building a solid foundation for fine-grained manipulation on open contact-rich tasks. The codebase and model checkpoints will be made publicly available to foster further research and development in tactile-enabled robotic manipulation.