TWIST2：可扩展、可移植且全面的人形机器人数据采集系统

摘要

大规模数据已推动机器人技术实现突破，从语言模型发展到双手操作中的视觉-语言-动作模型。然而仿人机器人领域仍缺乏同等高效的数据采集框架。现有仿人遥操作系统要么采用解耦控制，要么依赖昂贵的动作捕捉设备。我们推出TWIST2——一种便携、无需动作捕捉的仿人遥操作与数据采集系统，在保持全身完整控制的同时提升可扩展性。该系统利用PICO4U VR获取实时人体全身运动数据，并通过成本约250美元的定制二自由度机器人颈部实现以自我为中心的视觉感知，从而达成全方位的人体至仿人机器人控制。我们展示了仿人机器人执行长时序灵巧移动任务的能力，可在15分钟内采集100组演示数据且成功率接近100%。基于此技术路径，我们提出分层视觉运动策略框架，能够基于第一视角视觉自主控制仿人全身。我们的视觉运动策略成功实现了全身灵巧操控与动态踢球任务。整个系统完全可复现并已在https://yanjieze.com/TWIST2 开源，采集的数据集也发布于https://twist-data.github.io。

English

Large-scale data has driven breakthroughs in robotics, from language models to vision-language-action models in bimanual manipulation. However, humanoid robotics lacks equally effective data collection frameworks. Existing humanoid teleoperation systems either use decoupled control or depend on expensive motion capture setups. We introduce TWIST2, a portable, mocap-free humanoid teleoperation and data collection system that preserves full whole-body control while advancing scalability. Our system leverages PICO4U VR for obtaining real-time whole-body human motions, with a custom 2-DoF robot neck (cost around $250) for egocentric vision, enabling holistic human-to-humanoid control. We demonstrate long-horizon dexterous and mobile humanoid skills and we can collect 100 demonstrations in 15 minutes with an almost 100% success rate. Building on this pipeline, we propose a hierarchical visuomotor policy framework that autonomously controls the full humanoid body based on egocentric vision. Our visuomotor policy successfully demonstrates whole-body dexterous manipulation and dynamic kicking tasks. The entire system is fully reproducible and open-sourced at https://yanjieze.com/TWIST2 . Our collected dataset is also open-sourced at https://twist-data.github.io .

TWIST2：可扩展、可移植且全面的人形机器人数据采集系统

TWIST2: Scalable, Portable, and Holistic Humanoid Data Collection System

摘要

Support