TANGO：以全身視覺-語言-動作模型於雜亂環境中進行人形導航

摘要

我們研究使用人形機器人在雜亂室內環境中導航的問題。與將導航建模為二維路徑規劃問題的傳統方法不同，人形機器人在雜亂環境中的穿越需要持續的幾何感知全身適應，包括協調的手臂放置、軀幹調整和步態調變，以在複雜的三維空間中進行無碰撞移動。我們介紹了 TANGO，這是第一個用於在雜亂環境中進行語言條件人形穿越的全身視覺語言導航框架。給定自然語言指令和以自我為中心的RGB觀測，TANGO 直接預測 29 自由度關節空間動作，用於下游全身控制。我們完全在模擬中訓練 TANGO，通過全局路徑規劃、運動學全身運動生成、障礙物感知運動編輯和基於強化學習的追蹤來合成多樣化的無碰撞穿越行為。該流程為學習語言條件全身策略提供了動態可行的動作監督。在大量的模擬實驗中，TANGO 在視覺語言導航中展示了最先進的性能，同時在需要越障的挑戰性場景中導航時優於強大的模組化基線。最後，我們將 TANGO 零樣本部署在 Unitree G1 人形機器人上，並在雜亂的真實世界場景中觀察到穩健的語言引導穿越，而無需在任何真實世界導航數據上進行訓練。

English

We study the problem of navigating cluttered indoor environments with a humanoid robot. Unlike conventional methods that model navigation as a 2D path planning problem, humanoid traversal in cluttered environments requires continuous geometry-aware whole-body adaptation, including coordinated arm placement, torso adjustment, and gait modulation for collision-free movement through complex 3D spaces. We introduce TANGO, the first whole-body vision-language navigation framework for language-conditioned humanoid traversal in cluttered environments. Given a natural-language instruction and egocentric RGB observations, TANGO directly predicts 29-DoF joint-space actions for downstream whole-body control. We train TANGO entirely in simulation by synthesizing diverse collision-free traversal behaviors via global path planning, kinematic whole-body motion generation, obstacle-aware motion editing, and RL-based tracking. This pipeline provides dynamically feasible action supervision for learning language-conditioned whole-body policies. In extensive simulation experiments, TANGO demonstrates state-of-the-art performance in vision-language navigation, while outperforming strong modular baselines in navigating challenging scenes requiring obstacle negotiation. Lastly, we deploy TANGO zero-shot on a Unitree G1 humanoid robot, and observe robust language-guided traversal in cluttered real-world scenes without training on any real-world navigation data.