DexUMI：以人手為通用操控介面實現靈巧操作

摘要

我们提出DexUMI——一种利用人手作为自然界面，将灵巧操作技能转移至多种机器人手的数据收集与策略学习框架。DexUMI包含硬件与软件两方面的适配，旨在最小化人手与各类机器人手之间的实体差异。硬件适配通过穿戴式手部外骨骼弥合运动学差距，不仅能在数据收集中提供直接的触觉反馈，还能将人体动作调整为机器人手可行的运动模式。软件适配则通过高保真机器人手图像修复技术，在视频数据中替换人手，以消除视觉差异。我们通过在两种不同的灵巧机器人手硬件平台上进行全面的现实世界实验，展示了DexUMI的能力，平均任务成功率达到了86%。

English

We present DexUMI - a data collection and policy learning framework that uses the human hand as the natural interface to transfer dexterous manipulation skills to various robot hands. DexUMI includes hardware and software adaptations to minimize the embodiment gap between the human hand and various robot hands. The hardware adaptation bridges the kinematics gap using a wearable hand exoskeleton. It allows direct haptic feedback in manipulation data collection and adapts human motion to feasible robot hand motion. The software adaptation bridges the visual gap by replacing the human hand in video data with high-fidelity robot hand inpainting. We demonstrate DexUMI's capabilities through comprehensive real-world experiments on two different dexterous robot hand hardware platforms, achieving an average task success rate of 86%.

DexUMI：以人手為通用操控介面實現靈巧操作

DexUMI: Using Human Hand as the Universal Manipulation Interface for Dexterous Manipulation

摘要

Support