DexNDM：通过关节级神经动力学模型弥合灵巧手内旋转的现实差距

DexNDM: Closing the Reality Gap for Dexterous In-Hand Rotation via Joint-Wise Neural Dynamics Model

October 9, 2025

作者: Xueyi Liu, He Wang, Li Yi

cs.AI

摘要

实现通用的手内物体旋转仍然是机器人学中的一大挑战，这主要源于将策略从仿真环境迁移至现实世界的困难。灵巧操作中复杂且接触密集的动力学特性造成了“现实鸿沟”，使得先前的研究仅限于涉及简单几何形状、有限物体尺寸和长宽比、受限手腕姿态或定制化手部的约束场景。我们通过一种新颖的框架应对这一仿真到现实的挑战，该框架使得在仿真中训练的单条策略能够泛化到现实世界中多种多样的物体和条件下。我们方法的核心在于一个关节级动力学模型，它通过学习有效拟合有限量的现实世界采集数据，并据此调整仿真策略的动作，从而弥合现实鸿沟。该模型具有极高的数据效率，并能通过将动力学分解到各个关节、将系统级影响压缩为低维变量，以及从每个关节自身的动态特性中学习其演变，隐式捕捉这些综合效应，从而在不同全手交互分布间实现泛化。我们将其与一套全自动数据收集策略相结合，该策略以最少的人工干预收集多样化的现实世界交互数据。我们的完整流程展现了前所未有的泛化能力：单条策略成功旋转了具有复杂形状（如动物）、高长宽比（高达5.33）和小尺寸的挑战性物体，同时处理了多样的手腕朝向和旋转轴。全面的现实世界评估及针对复杂任务的遥操作应用验证了我们方法的有效性和鲁棒性。网站：https://meowuu7.github.io/DexNDM/

English

Achieving generalized in-hand object rotation remains a significant challenge in robotics, largely due to the difficulty of transferring policies from simulation to the real world. The complex, contact-rich dynamics of dexterous manipulation create a "reality gap" that has limited prior work to constrained scenarios involving simple geometries, limited object sizes and aspect ratios, constrained wrist poses, or customized hands. We address this sim-to-real challenge with a novel framework that enables a single policy, trained in simulation, to generalize to a wide variety of objects and conditions in the real world. The core of our method is a joint-wise dynamics model that learns to bridge the reality gap by effectively fitting limited amount of real-world collected data and then adapting the sim policy's actions accordingly. The model is highly data-efficient and generalizable across different whole-hand interaction distributions by factorizing dynamics across joints, compressing system-wide influences into low-dimensional variables, and learning each joint's evolution from its own dynamic profile, implicitly capturing these net effects. We pair this with a fully autonomous data collection strategy that gathers diverse, real-world interaction data with minimal human intervention. Our complete pipeline demonstrates unprecedented generality: a single policy successfully rotates challenging objects with complex shapes (e.g., animals), high aspect ratios (up to 5.33), and small sizes, all while handling diverse wrist orientations and rotation axes. Comprehensive real-world evaluations and a teleoperation application for complex tasks validate the effectiveness and robustness of our approach. Website: https://meowuu7.github.io/DexNDM/