Play2Perfect：灵巧操作预训练中什么对精确装配最关键？

摘要

多指灵巧手有望实现人类双手般的速度与灵巧性，但诸如精密装配等难题至今仍难以攻克。这类任务具有高接触性特征，使得通过模仿学习进行数据收集变得困难；同时其奖励信号稀疏，导致直接采用强化学习进行探索难以奏效。因此，先前的研究通过采用专用夹爪、工具附件和环境固定装置来结构化问题，取得了阶段性进展。本研究提出观点：在机器人掌握精密装配之前，必须首先学会"玩耍"。我们进一步探究：玩耍学习过程中的哪些因素对精密装配至关重要？为此提出Play2Perfect框架——一种通过多样化物体与目标进行任务无关式玩耍预训练的强化学习方案，后续再针对精密装配进行精炼。玩耍阶段的目标是获取可复用的操作先验知识（如抓取、手内重定向、位姿逼近），微调阶段则将这些通用先验知识适配到装配任务，将探索聚焦于最终决定成败的高接触、高精度交互环节。我们系统研究了玩耍预训练中的关键设计选择（包括物体多样性、训练目标、轨迹多样性及目标精度），证明即使提供密集的多阶段奖励，该先验知识的样本效率仍比从零开始的强化学习高出33倍。我们实现了零样本仿真到现实迁移，在仅0.5毫米接触间隙的紧配合插入任务中达到60%成功率，在多部件长时序装配与拧螺丝任务中成功率超过50%。

English

Multi-fingered robots promise the speed and dexterity of human hands, yet challenging problems such as precise assembly have remained out of reach. These tasks are contact-rich, making data collection for imitation learning difficult, and sparse-reward, making direct exploration with reinforcement learning (RL) intractable. Consequently, prior work has made progress by structuring the problem with specialized grippers, tool attachments, and environment fixtures. In this work, we argue that before a robot can perfect precise assembly, it must first learn to play. We further ask the question: what factors in the process of learning to play matter for precise assembly? We propose Play2Perfect, an RL framework for task-agnostic pretraining through play on diverse objects and goals, which is then perfected on precise assembly. The goal of play is to acquire reusable manipulation priors, such as grasping, in-hand reorientation and pose reaching. Finetuning then adapts this general prior to assembly, focusing exploration on the final contact-rich, high-precision interactions needed for success. We systematically study key design choices in play pretraining, including object diversity, training objective, trajectory diversity, and goal precision. We show that our prior is 33x more sample-efficient than RL training from scratch, even when provided with dense, multi-stage rewards. We demonstrate zero-shot sim-to-real transfer, achieving 60% success on tight insertions with only 0.5 mm contact clearance, and over 50% success on long-horizon multi-part assembly and screwing.