Play2Perfect：靈巧操作預訓練中何種因素對精密裝配至關重要？

摘要

多指機器人承諾擁有如人手的靈巧速度與靈活性，但諸如精密組裝等具挑戰性的問題仍難以實現。此類任務接觸密集，導致模仿學習的數據收集困難；且獎勵稀疏，使得直接利用強化學習進行探索變得不可行。因此，先前的研究透過以專用夾具、工具附件及環境固定裝置來結構化問題，從而取得進展。在本研究中，我們主張：在機器人能夠完善精密組裝之前，必須先學會玩耍。我們進一步探討：在學習玩耍的過程中，哪些因素對精密組裝至關重要？我們提出Play2Perfect，一個透過在多樣物體與目標上進行玩耍來實現任務無關預訓練的強化學習框架，之後再針對精密組裝進行微調優化。玩耍的目標是獲取可重複使用的操作先驗，例如抓取、手中重新定位及姿態達成。微調則將此通用先驗適應於組裝任務，將探索聚焦於最終需要成功的接觸密集、高精度互動。我們系統性地研究了玩耍預訓練中的關鍵設計選擇，包括物體多樣性、訓練目標、軌跡多樣性及目標精度。結果顯示，即使提供了密集的多階段獎勵，我們的先驗在樣本效率上仍比從零開始的強化學習高出33倍。我們展示了零樣本從模擬到真實的轉移，在僅有0.5毫米接觸間隙的緊密插入任務中達到60%的成功率，並在長時序多零件組裝與螺絲鎖付任務中達到超過50%的成功率。

English

Multi-fingered robots promise the speed and dexterity of human hands, yet challenging problems such as precise assembly have remained out of reach. These tasks are contact-rich, making data collection for imitation learning difficult, and sparse-reward, making direct exploration with reinforcement learning (RL) intractable. Consequently, prior work has made progress by structuring the problem with specialized grippers, tool attachments, and environment fixtures. In this work, we argue that before a robot can perfect precise assembly, it must first learn to play. We further ask the question: what factors in the process of learning to play matter for precise assembly? We propose Play2Perfect, an RL framework for task-agnostic pretraining through play on diverse objects and goals, which is then perfected on precise assembly. The goal of play is to acquire reusable manipulation priors, such as grasping, in-hand reorientation and pose reaching. Finetuning then adapts this general prior to assembly, focusing exploration on the final contact-rich, high-precision interactions needed for success. We systematically study key design choices in play pretraining, including object diversity, training objective, trajectory diversity, and goal precision. We show that our prior is 33x more sample-efficient than RL training from scratch, even when provided with dense, multi-stage rewards. We demonstrate zero-shot sim-to-real transfer, achieving 60% success on tight insertions with only 0.5 mm contact clearance, and over 50% success on long-horizon multi-part assembly and screwing.