OASIS：從模擬數據收集到真實世界的人形機器人移動操作

摘要

近年來，機器人操作領域的進展主要得益於大規模示範學習。然而，針對人形機器人的全身操控任務，現有數據來源在軌跡品質與可擴展性之間存在難以調和的取捨。真實世界的遙操作能提供最高品質的軌跡，但需要專用的物理空間與耗時的場景重置。模擬則為此困境提供替代方案：無需實體硬體，即可大規模生成乾淨且符合本體結構的數據。本文提出OASIS，這是一個基於模擬數據的人形機器人全身操控框架。OASIS利用3D生成模型，從真實世界影像中自動重建逼真的物體資產。在此基礎上，先於模擬環境中透過遙操作收集軌跡，再於後處理階段對其進行多樣化域隨機化的擴增。憑藉所產生的模擬數據，我們進一步設計了一套分層式視覺運動策略，應用於人形機器人的全身操控。在真實人形機器人上的大量實驗顯示，在零樣本部署條件下，基於模擬數據訓練的策略在多數任務中成功率均高於基於真實機器人遙操作數據訓練的策略。這主要歸因於模擬渲染涵蓋了廣泛的光照與環境變化，而真實機器人數據無法捕捉這些變化。專案頁面請見 https://oasis-humanoid.github.io/。

English

Recent progress in robot manipulation has been largely driven by learning from large-scale demonstrations. For humanoid robot loco-manipulation tasks, however, existing data sources force an unsatisfying tradeoff between trajectory quality and scalability. Real-world teleoperation provides the highest-quality trajectories but requires dedicated physical space and time-consuming scene resets. Simulation offers an alternative way out of this dilemma: it can produce clean, embodiment-aligned data at scale without any physical hardware. In this paper, we propose OASIS, a simulation-data-driven framework for humanoid loco-manipulation. OASIS automatically reconstructs realistic object assets from real-world images using a 3D generative model. Based on these assets, trajectories are first collected through teleoperation in simulation, and then augmented under diverse domain randomizations in a post-processing stage. With the resulting simulation data, we further design a hierarchical visuomotor policy for humanoid loco-manipulation. Extensive experiments on the real humanoid robot show that, under zero-shot deployment, the policy trained on our simulation data achieves higher success rates on most tasks than that trained on real-robot teleoperation data, owing largely to the broad lighting and environmental variations covered by our simulation rendering, which real-robot data fails to capture. The project page is available at https://oasis-humanoid.github.io/.