具有改进的三维扩散策略的通用人形操作

Generalizable Humanoid Manipulation with Improved 3D Diffusion Policies

October 14, 2024

作者: Yanjie Ze, Zixuan Chen, Wenhao Wang, Tianyi Chen, Xialin He, Ying Yuan, Xue Bin Peng, Jiajun Wu

cs.AI

摘要

长期以来，能够在各种环境中自主运行的人形机器人一直是机器人学家的目标。然而，人形机器人的自主操作能力在自主操作方面主要受限于一种特定场景，主要是由于获取通用技能的困难。最近在3D视觉运动策略方面的进展，如3D扩散策略（DP3），显示出将这些能力拓展到更广泛环境中的潜力。然而，3D视觉运动策略通常依赖于摄像头校准和点云分割，这对于在人形机器人等移动机器人上部署提出了挑战。在这项工作中，我们介绍了改进的3D扩散策略（iDP3），这是一种新颖的3D视觉运动策略，通过利用自我中心的3D视觉表示来消除这些约束。我们展示了iDP3使得一个全尺寸的人形机器人能够在各种真实场景中自主执行技能，仅使用在实验室中收集的数据。视频可在以下链接中查看：https://humanoid-manipulation.github.io

English

Humanoid robots capable of autonomous operation in diverse environments have long been a goal for roboticists. However, autonomous manipulation by humanoid robots has largely been restricted to one specific scene, primarily due to the difficulty of acquiring generalizable skills. Recent advances in 3D visuomotor policies, such as the 3D Diffusion Policy (DP3), have shown promise in extending these capabilities to wilder environments. However, 3D visuomotor policies often rely on camera calibration and point-cloud segmentation, which present challenges for deployment on mobile robots like humanoids. In this work, we introduce the Improved 3D Diffusion Policy (iDP3), a novel 3D visuomotor policy that eliminates these constraints by leveraging egocentric 3D visual representations. We demonstrate that iDP3 enables a full-sized humanoid robot to autonomously perform skills in diverse real-world scenarios, using only data collected in the lab. Videos are available at: https://humanoid-manipulation.github.io