RoboCook：使用多樣工具進行長時間視野的彈塑性物體操作

RoboCook: Long-Horizon Elasto-Plastic Object Manipulation with Diverse Tools

June 26, 2023

作者: Haochen Shi, Huazhe Xu, Samuel Clarke, Yunzhu Li, Jiajun Wu

cs.AI

摘要

人類在複雜的長期軟體操作任務中表現出色，透過靈活的工具使用：烘培麵包需要用刀切割麵團，用擀麵棍擀平。被視為人類認知的標誌，自主機器人中的工具使用仍然受限於理解工具-物體互動的挑戰。在這裡，我們開發了一個智能機器人系統 RoboCook，它能感知、建模和操作具有不同工具的彈塑性物體。RoboCook 使用點雲場景表示，用圖神經網絡 (GNNs) 建模工具-物體互動，並結合工具分類與自監督策略學習來制定操作計劃。我們展示，通過僅僅 20 分鐘的真實世界互動數據，一臺通用機器人手臂可以學會複雜的長期軟體物體操作任務，例如製作餃子和字母曲奇餅乾。廣泛的評估顯示，RoboCook 顯著優於最先進的方法，具有抵抗嚴重外部干擾的穩健性，並展現對不同材料的適應能力。

English

Humans excel in complex long-horizon soft body manipulation tasks via flexible tool use: bread baking requires a knife to slice the dough and a rolling pin to flatten it. Often regarded as a hallmark of human cognition, tool use in autonomous robots remains limited due to challenges in understanding tool-object interactions. Here we develop an intelligent robotic system, RoboCook, which perceives, models, and manipulates elasto-plastic objects with various tools. RoboCook uses point cloud scene representations, models tool-object interactions with Graph Neural Networks (GNNs), and combines tool classification with self-supervised policy learning to devise manipulation plans. We demonstrate that from just 20 minutes of real-world interaction data per tool, a general-purpose robot arm can learn complex long-horizon soft object manipulation tasks, such as making dumplings and alphabet letter cookies. Extensive evaluations show that RoboCook substantially outperforms state-of-the-art approaches, exhibits robustness against severe external disturbances, and demonstrates adaptability to different materials.