ManipTrans：基於殘差學習的高效靈巧雙手操作遷移

ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learning

March 27, 2025

作者: Kailin Li, Puhao Li, Tengyu Liu, Yuyang Li, Siyuan Huang

cs.AI

摘要

人類雙手在互動中扮演核心角色，這促使靈巧機器人操控的研究日益增加。數據驅動的具身人工智慧算法需要精確、大規模且類似人類的操控序列，而這些序列透過傳統的強化學習或現實世界的遙控操作難以獲得。為解決這一問題，我們提出了ManipTrans，這是一種新穎的兩階段方法，用於在模擬環境中高效地將人類雙手技能轉移至靈巧機器人手上。ManipTrans首先預訓練一個通用軌跡模仿器來模仿手部動作，然後在互動約束下微調特定的殘差模組，從而實現複雜雙手任務的高效學習與精確執行。實驗表明，ManipTrans在成功率、逼真度和效率上均超越了現有最先進的方法。利用ManipTrans，我們將多個手物互動數據集轉移至機器人手上，創建了DexManipNet，這是一個大規模數據集，涵蓋了如筆帽蓋合和瓶蓋旋開等先前未探索的任務。DexManipNet包含3,300個機器人操控片段，且易於擴展，為靈巧手的策略訓練提供了便利，並支持實際應用的部署。

English

Human hands play a central role in interacting, motivating increasing research in dexterous robotic manipulation. Data-driven embodied AI algorithms demand precise, large-scale, human-like manipulation sequences, which are challenging to obtain with conventional reinforcement learning or real-world teleoperation. To address this, we introduce ManipTrans, a novel two-stage method for efficiently transferring human bimanual skills to dexterous robotic hands in simulation. ManipTrans first pre-trains a generalist trajectory imitator to mimic hand motion, then fine-tunes a specific residual module under interaction constraints, enabling efficient learning and accurate execution of complex bimanual tasks. Experiments show that ManipTrans surpasses state-of-the-art methods in success rate, fidelity, and efficiency. Leveraging ManipTrans, we transfer multiple hand-object datasets to robotic hands, creating DexManipNet, a large-scale dataset featuring previously unexplored tasks like pen capping and bottle unscrewing. DexManipNet comprises 3.3K episodes of robotic manipulation and is easily extensible, facilitating further policy training for dexterous hands and enabling real-world deployments.