DexNDM：通過關節級神經動力學模型縮小靈巧手內旋轉的現實差距

DexNDM: Closing the Reality Gap for Dexterous In-Hand Rotation via Joint-Wise Neural Dynamics Model

October 9, 2025

作者: Xueyi Liu, He Wang, Li Yi

cs.AI

摘要

實現通用的手內物體旋轉仍然是機器人領域的一個重大挑戰，這主要源於將策略從模擬環境轉移到現實世界的困難。靈巧操作中複雜且接觸密集的動力學特性造成了「現實差距」，這使得先前的研究僅限於涉及簡單幾何形狀、有限物體尺寸和長寬比、受限手腕姿勢或定制化機械手的約束場景。我們提出了一種新穎的框架來應對這一模擬到現實的挑戰，該框架使在模擬中訓練的單一策略能夠泛化到現實世界中的多種物體和條件。我們方法的核心是一個關節級別的動力學模型，該模型通過有效擬合有限的現實世界收集數據來學習彌合現實差距，並據此調整模擬策略的行動。該模型具有高度的數據效率，並通過將動力學分解到各個關節、將系統範圍的影響壓縮到低維變量中，以及從每個關節自身的動態特性中學習其演化，從而隱式地捕捉這些淨效應，實現了跨不同全手交互分佈的泛化能力。我們將此與一種完全自主的數據收集策略相結合，該策略以最少的人為干預收集多樣化的現實世界交互數據。我們的完整流程展示了前所未有的通用性：單一策略成功旋轉了具有複雜形狀（如動物）、高長寬比（高達5.33）和小尺寸的挑戰性物體，同時處理了多樣的手腕方向和旋轉軸。全面的現實世界評估和用於複雜任務的遙操作應用驗證了我們方法的有效性和魯棒性。網站：https://meowuu7.github.io/DexNDM/

English

Achieving generalized in-hand object rotation remains a significant challenge in robotics, largely due to the difficulty of transferring policies from simulation to the real world. The complex, contact-rich dynamics of dexterous manipulation create a "reality gap" that has limited prior work to constrained scenarios involving simple geometries, limited object sizes and aspect ratios, constrained wrist poses, or customized hands. We address this sim-to-real challenge with a novel framework that enables a single policy, trained in simulation, to generalize to a wide variety of objects and conditions in the real world. The core of our method is a joint-wise dynamics model that learns to bridge the reality gap by effectively fitting limited amount of real-world collected data and then adapting the sim policy's actions accordingly. The model is highly data-efficient and generalizable across different whole-hand interaction distributions by factorizing dynamics across joints, compressing system-wide influences into low-dimensional variables, and learning each joint's evolution from its own dynamic profile, implicitly capturing these net effects. We pair this with a fully autonomous data collection strategy that gathers diverse, real-world interaction data with minimal human intervention. Our complete pipeline demonstrates unprecedented generality: a single policy successfully rotates challenging objects with complex shapes (e.g., animals), high aspect ratios (up to 5.33), and small sizes, all while handling diverse wrist orientations and rotation axes. Comprehensive real-world evaluations and a teleoperation application for complex tasks validate the effectiveness and robustness of our approach. Website: https://meowuu7.github.io/DexNDM/