作为桥接行为的翻译：将操作技能从人类迁移到机器人

摘要

我们研究能否从人类动作中习得新的操作技能，并将其迁移至配备平行夹爪的双臂机器人。人类动作数据成本低廉、来源丰富且形式多样，使其成为扩展机器人学习最具潜力的资源之一。然而，将技能从人类迁移至机器人仍面临挑战：大多数先前研究将人类视为另一种双臂六自由度（6DoF）实体，但手部姿态估计存在噪声，且人类手指的接触模式与平行夹爪存在本质差异。我们认为，从人类数据中学习包含旋转信息的行为信号并非最优方案，因此提出一种桥接动作表征：以初始头部相机坐标系为基准的相对腕部平移——这是人类与机器人共享的动作空间。为应对不同实体中可能缺失某些动作分量的情况，我们构建了类似π_0的视觉-语言-动作模型，采用交错动作标记与注意力掩码机制。在一系列新型双臂操作任务中，我们的桥接动作表征将人类操作知识迁移至机器人的效果远超含噪的六自由度人类动作，且性能随人类数据量增长而提升。

English

We study whether we can learn novel manipulation skills from human actions to a bi-manual robot with parallel grippers. Human action data is cheap, abundant, and diverse, making it one of the most promising resources for scaling up robot learning. Yet transferring skills from humans to robots remains hard: most prior work treats humans as just another bi-manual 6DoF embodiment, where hand-pose estimates are noisy and the contact patterns of human fingers differ fundamentally from those of a parallel gripper. We argue that learning rotation-inclusive action signals from human data is therefore sub-optimal, and instead propose a bridging action representation: the relative wrist translation within the initial head-camera frame, an action space shared by humans and robots. To handle the potential absence of certain action components in different embodiments, we build a π_0-like vision-language-action model with interleaved action tokens and attention masking. On a suite of novel bi-manual manipulation tasks, our bridging action transfers human manipulation knowledge to robots far more effectively than noisy 6DoF human actions and scales with the amount of human data.