具有改進的三維擴散策略的通用人形機器人操作

Generalizable Humanoid Manipulation with Improved 3D Diffusion Policies

October 14, 2024

作者: Yanjie Ze, Zixuan Chen, Wenhao Wang, Tianyi Chen, Xialin He, Ying Yuan, Xue Bin Peng, Jiajun Wu

cs.AI

摘要

長久以來，具備在各種環境中自主運作能力的人形機器人一直是機器人學家的目標。然而，人形機器人的自主操作能力在大多數情況下被限制在特定場景，主要是由於獲取通用技能的困難。最近在3D視覺運動策略方面的進展，例如3D擴散策略（DP3），展示了將這些能力擴展到更廣泛環境的潛力。然而，3D視覺運動策略通常依賴攝像機校準和點雲分割，這對於在人形機器人等移動機器人上部署提出了挑戰。在這項工作中，我們介紹了改進的3D擴散策略（iDP3），這是一種新穎的3D視覺運動策略，通過利用自我中心的3D視覺表示來消除這些限制。我們展示了iDP3使一個全尺寸的人形機器人能夠在各種真實場景中自主執行技能，僅使用在實驗室收集的數據。視頻可在以下網址查看：https://humanoid-manipulation.github.io

English

Humanoid robots capable of autonomous operation in diverse environments have long been a goal for roboticists. However, autonomous manipulation by humanoid robots has largely been restricted to one specific scene, primarily due to the difficulty of acquiring generalizable skills. Recent advances in 3D visuomotor policies, such as the 3D Diffusion Policy (DP3), have shown promise in extending these capabilities to wilder environments. However, 3D visuomotor policies often rely on camera calibration and point-cloud segmentation, which present challenges for deployment on mobile robots like humanoids. In this work, we introduce the Improved 3D Diffusion Policy (iDP3), a novel 3D visuomotor policy that eliminates these constraints by leveraging egocentric 3D visual representations. We demonstrate that iDP3 enables a full-sized humanoid robot to autonomously perform skills in diverse real-world scenarios, using only data collected in the lab. Videos are available at: https://humanoid-manipulation.github.io