OASIS：从仿真数据收集到现实世界人形机器人移动操控

摘要

近期，机器人操作领域的进展主要得益于大规模示范学习。然而，对于人形机器人的定位操作任务，现有数据源在轨迹质量与可扩展性之间难以两全：真实世界远程操作虽能提供最高质量的轨迹，却需要专用的物理空间和耗时的场景重置；而模拟环境则为此困境提供了另一条出路——无需物理硬件即可大规模生成与实体对齐的干净数据。本文提出OASIS框架，一种基于模拟数据的面向人形机器人定位操作的方案。该框架利用3D生成模型从真实世界图像中自动重建逼真的物体资产，并基于这些资产先在模拟环境中通过远程操作采集轨迹，再在后处理阶段通过多样化的域随机化增强数据。基于生成的模拟数据，我们进一步设计了用于人形机器人定位操作的分层视觉运动策略。在真实人形机器人上的大量实验表明，零样本部署条件下，基于模拟数据训练的策略在多数任务上成功率高于基于真实机器人远程操作数据训练的策略，这主要得益于模拟渲染所覆盖的广泛光照与环境变化——这是真实机器人数据难以企及的。项目页面见 https://oasis-humanoid.github.io/。

English

Recent progress in robot manipulation has been largely driven by learning from large-scale demonstrations. For humanoid robot loco-manipulation tasks, however, existing data sources force an unsatisfying tradeoff between trajectory quality and scalability. Real-world teleoperation provides the highest-quality trajectories but requires dedicated physical space and time-consuming scene resets. Simulation offers an alternative way out of this dilemma: it can produce clean, embodiment-aligned data at scale without any physical hardware. In this paper, we propose OASIS, a simulation-data-driven framework for humanoid loco-manipulation. OASIS automatically reconstructs realistic object assets from real-world images using a 3D generative model. Based on these assets, trajectories are first collected through teleoperation in simulation, and then augmented under diverse domain randomizations in a post-processing stage. With the resulting simulation data, we further design a hierarchical visuomotor policy for humanoid loco-manipulation. Extensive experiments on the real humanoid robot show that, under zero-shot deployment, the policy trained on our simulation data achieves higher success rates on most tasks than that trained on real-robot teleoperation data, owing largely to the broad lighting and environmental variations covered by our simulation rendering, which real-robot data fails to capture. The project page is available at https://oasis-humanoid.github.io/.