DexUMI：以人手作为通用操控界面的灵巧操作技术

摘要

我们推出DexUMI——一个利用人手作为自然接口的数据收集与策略学习框架，旨在将灵巧操作技能迁移至多种机器人手。DexUMI包含硬件与软件适配，以最小化人手与各类机器人手之间的实体差异。硬件适配通过穿戴式手部外骨骼桥接运动学差距，不仅允许在数据收集中直接提供触觉反馈，还将人类动作调整为机器人手可执行的动作。软件适配则通过高保真机器人手图像修复技术，在视频数据中替换人手，从而弥合视觉差异。我们通过在两种不同的灵巧机器人手硬件平台上进行全面的真实世界实验，展示了DexUMI的能力，平均任务成功率达到了86%。

English

We present DexUMI - a data collection and policy learning framework that uses the human hand as the natural interface to transfer dexterous manipulation skills to various robot hands. DexUMI includes hardware and software adaptations to minimize the embodiment gap between the human hand and various robot hands. The hardware adaptation bridges the kinematics gap using a wearable hand exoskeleton. It allows direct haptic feedback in manipulation data collection and adapts human motion to feasible robot hand motion. The software adaptation bridges the visual gap by replacing the human hand in video data with high-fidelity robot hand inpainting. We demonstrate DexUMI's capabilities through comprehensive real-world experiments on two different dexterous robot hand hardware platforms, achieving an average task success rate of 86%.

DexUMI：以人手作为通用操控界面的灵巧操作技术

DexUMI: Using Human Hand as the Universal Manipulation Interface for Dexterous Manipulation

摘要

Summary

Support

Support