DexJoCo：基於MuJoCo的任務導向靈巧操作基準與工具包

摘要

實現人類等級的操作能力需要具備靈巧機器人手部，以執行複雜的物體互動。為進一步提升此類能力，需建立標準化基準以進行系統性評估。然而，現有的靈巧操作基準缺乏能反映靈巧手部相較於平行夾爪之獨特操作能力的任務，亦缺少完整的評估流程。本文提出 DexJoCo——一套專注於任務導向靈巧操作的基準與工具包，包含 11 項以功能為基礎的任務，用以評估工具使用、雙手協調、長時域執行及推理能力。我們開發了一套低成本數據收集系統，並在這些任務中收集了 1,100 條軌跡，同時支援域隨機化以評估其穩健性。我們在多種設定下（包含視覺與動力學隨機化、多任務訓練、以及動作頭適配）對現代模型進行基準測試。透過廣泛的實證分析，我們辨識出當前靈巧操作策略中的數項重要見解與常見限制，並凸顯出未來靈巧手部機器人學習研究的關鍵挑戰。專案頁面請見：https://dexjoco.github.io

English

Achieving human-level manipulation requires dexterous robotic hands capable of complex object interactions. Advancing such capabilities further demands standardized benchmarks for systematic evaluation. However, existing dexterous benchmarks lack tasks that reflect the unique manipulation capabilities of dexterous hands over parallel grippers, as well as comprehensive evaluation pipelines. In this paper, we present DexJoCo, a benchmark and toolkit for task-oriented dexterous manipulation, comprising 11 functionally grounded tasks that evaluate tool-use, bimanual coordination, long-horizon execution, and reasoning. We develop a low-cost data collection system and collect 1.1K trajectories across these tasks, with support for domain randomization to assess robustness. We benchmark modern models under diverse settings, including visual and dynamics randomization, multi-task training, and action-head adaptation. Through extensive empirical analysis, we identify several important insights and common limitations of current policies in dexterous manipulation, highlighting key challenges for future research in dexterous hand robot learning. Project page available at: https://dexjoco.github.io