DexJoCo：基于MuJoCo的面向任务的灵巧操作基准与工具包

摘要

实现人类级操作需要具备复杂物体交互能力的灵巧机械手。为进一步推进此类能力发展，需要建立标准化基准进行系统评估。然而，现有灵巧操作基准既缺乏能体现灵巧手相较于平行夹爪独特操作能力的任务，也缺少完整的评估流程。本文提出DexJoCo——面向任务导向型灵巧操作的基准与工具包，包含11项功能型任务，评估工具使用、双手协调、长时域执行及推理能力。我们开发了低成本数据采集系统，收集了涵盖这些任务的1100条轨迹，并支持域随机化以评估鲁棒性。我们在多样化设置下对现代模型进行基准测试，包括视觉与动力学随机化、多任务训练及动作头适配。通过广泛实验分析，我们揭示了当前灵巧操作策略中若干重要发现与常见局限，指出了灵巧手机器人学习领域未来研究的关键挑战。项目页面：https://dexjoco.github.io

English

Achieving human-level manipulation requires dexterous robotic hands capable of complex object interactions. Advancing such capabilities further demands standardized benchmarks for systematic evaluation. However, existing dexterous benchmarks lack tasks that reflect the unique manipulation capabilities of dexterous hands over parallel grippers, as well as comprehensive evaluation pipelines. In this paper, we present DexJoCo, a benchmark and toolkit for task-oriented dexterous manipulation, comprising 11 functionally grounded tasks that evaluate tool-use, bimanual coordination, long-horizon execution, and reasoning. We develop a low-cost data collection system and collect 1.1K trajectories across these tasks, with support for domain randomization to assess robustness. We benchmark modern models under diverse settings, including visual and dynamics randomization, multi-task training, and action-head adaptation. Through extensive empirical analysis, we identify several important insights and common limitations of current policies in dexterous manipulation, highlighting key challenges for future research in dexterous hand robot learning. Project page available at: https://dexjoco.github.io