RoboCook：使用多样工具进行长时程弹塑性物体操作

摘要

人类在复杂的长期软体操纵任务中表现出色，通过灵活运用工具：面包烘焙需要用刀切割面团，用擀面杖擀平。工具使用被视为人类认知的标志，但在自主机器人中，由于理解工具-物体交互的挑战，其应用仍受限制。在这里，我们开发了一个智能机器人系统，RoboCook，它可以感知、建模和操纵具有不同工具的弹塑性物体。RoboCook使用点云场景表示，用图神经网络（GNNs）模拟工具-物体交互，并将工具分类与自监督策略学习相结合，制定操纵计划。我们展示，仅通过每种工具20分钟的真实世界互动数据，通用机器人手臂就能学会复杂的长期软体操纵任务，如制作饺子和字母饼干。广泛的评估表明，RoboCook明显优于最先进的方法，在面对严重外部干扰时表现出鲁棒性，并展示对不同材料的适应能力。

English

Humans excel in complex long-horizon soft body manipulation tasks via flexible tool use: bread baking requires a knife to slice the dough and a rolling pin to flatten it. Often regarded as a hallmark of human cognition, tool use in autonomous robots remains limited due to challenges in understanding tool-object interactions. Here we develop an intelligent robotic system, RoboCook, which perceives, models, and manipulates elasto-plastic objects with various tools. RoboCook uses point cloud scene representations, models tool-object interactions with Graph Neural Networks (GNNs), and combines tool classification with self-supervised policy learning to devise manipulation plans. We demonstrate that from just 20 minutes of real-world interaction data per tool, a general-purpose robot arm can learn complex long-horizon soft object manipulation tasks, such as making dumplings and alphabet letter cookies. Extensive evaluations show that RoboCook substantially outperforms state-of-the-art approaches, exhibits robustness against severe external disturbances, and demonstrates adaptability to different materials.

RoboCook：使用多样工具进行长时程弹塑性物体操作

RoboCook: Long-Horizon Elasto-Plastic Object Manipulation with Diverse Tools

摘要

Support