学习旋转“笔”技巧的经验

Lessons from Learning to Spin "Pens"

July 26, 2024

作者: Jun Wang, Ying Yuan, Haichuan Che, Haozhi Qi, Yi Ma, Jitendra Malik, Xiaolong Wang

cs.AI

摘要

在我们日常生活中，类似笔的物体的手部操作是一项重要技能，因为许多工具如锤子和螺丝刀形状类似。然而，由于缺乏高质量的演示以及模拟与真实世界之间存在显著差距，当前基于学习的方法在这项任务上面临困难。在这项工作中，我们通过展示旋转类似笔的物体的能力，推动了基于学习的手部操作系统的边界。我们首先使用强化学习训练具有特权信息的预言策略，并在模拟中生成高保真度的轨迹数据集。这有两个目的：1）在模拟中预训练感知运动策略；2）在真实世界中进行开环轨迹重放。然后，我们使用这些真实世界轨迹对感知运动策略进行微调，以使其适应真实世界的动态。通过不到50条轨迹，我们的策略学会旋转超过十种具有不同物理特性的类似笔的物体，实现多次旋转。我们对设计选择进行了全面分析，并分享了开发过程中的经验教训。

English

In-hand manipulation of pen-like objects is an important skill in our daily lives, as many tools such as hammers and screwdrivers are similarly shaped. However, current learning-based methods struggle with this task due to a lack of high-quality demonstrations and the significant gap between simulation and the real world. In this work, we push the boundaries of learning-based in-hand manipulation systems by demonstrating the capability to spin pen-like objects. We first use reinforcement learning to train an oracle policy with privileged information and generate a high-fidelity trajectory dataset in simulation. This serves two purposes: 1) pre-training a sensorimotor policy in simulation; 2) conducting open-loop trajectory replay in the real world. We then fine-tune the sensorimotor policy using these real-world trajectories to adapt it to the real world dynamics. With less than 50 trajectories, our policy learns to rotate more than ten pen-like objects with different physical properties for multiple revolutions. We present a comprehensive analysis of our design choices and share the lessons learned during development.