RP1M：一项钢琴演奏双手灵巧机器人大规模运动数据集

摘要

长期以来，赋予机器手人类水平的灵巧性一直是一个研究目标。双手机器人演奏钢琴构成了一个任务，结合了动态任务的挑战，如生成快速而精确的动作，以及较慢但接触丰富的操纵问题。虽然基于强化学习的方法在单一任务表现方面表现出有希望的结果，但这些方法在多首歌曲设置中面临困难。我们的工作旨在弥合这一差距，从而实现规模化的机器人钢琴演奏模仿学习方法。为此，我们引入了“机器人钢琴100万”（RP1M）数据集，其中包含超过一百万条双手机器人钢琴演奏运动数据轨迹。我们将手指放置形式化为最优输运问题，从而实现对大量未标记歌曲的自动注释。对现有的模仿学习方法进行基准测试表明，通过利用RP1M，这些方法达到了最先进的机器人钢琴演奏性能。

English

It has been a long-standing research goal to endow robot hands with human-level dexterity. Bi-manual robot piano playing constitutes a task that combines challenges from dynamic tasks, such as generating fast while precise motions, with slower but contact-rich manipulation problems. Although reinforcement learning based approaches have shown promising results in single-task performance, these methods struggle in a multi-song setting. Our work aims to close this gap and, thereby, enable imitation learning approaches for robot piano playing at scale. To this end, we introduce the Robot Piano 1 Million (RP1M) dataset, containing bi-manual robot piano playing motion data of more than one million trajectories. We formulate finger placements as an optimal transport problem, thus, enabling automatic annotation of vast amounts of unlabeled songs. Benchmarking existing imitation learning approaches shows that such approaches reach state-of-the-art robot piano playing performance by leveraging RP1M.

RP1M：一项钢琴演奏双手灵巧机器人大规模运动数据集

RP1M: A Large-Scale Motion Dataset for Piano Playing with Bi-Manual Dexterous Robot Hands

摘要

Summary

Support

Support