HomeRobot：开放词汇移动操作

HomeRobot: Open-Vocabulary Mobile Manipulation

June 20, 2023

作者: Sriram Yenamandra, Arun Ramachandran, Karmesh Yadav, Austin Wang, Mukul Khanna, Theophile Gervet, Tsung-Yen Yang, Vidhi Jain, Alexander William Clegg, John Turner, Zsolt Kira, Manolis Savva, Angel Chang, Devendra Singh Chaplot, Dhruv Batra, Roozbeh Mottaghi, Yonatan Bisk, Chris Paxton

cs.AI

摘要

HomeRobot（名词）：一种价格实惠的顺应性机器人，可在家中导航并操作各种物体，以完成日常任务。开放词汇移动操作（OVMM）是在任何未知环境中拾取任何物体并将其放置在指定位置的问题。这对于机器人成为人类环境中有用的助手是一个基础性挑战，因为它涉及解决来自机器人技术各个领域的子问题：感知、语言理解、导航和操作对OVMM都至关重要。此外，整合这些子问题的解决方案本身也带来重大挑战。为推动这一领域的研究，我们引入了HomeRobot OVMM基准测试，其中一个代理程序在家庭环境中导航，抓取新颖物体并将其放置在目标容器上。HomeRobot有两个组成部分：一个模拟组件，使用新的、高质量的多房间家庭环境中的大量多样化的物体集合；以及一个现实世界组件，提供一个针对低成本Hello Robot Stretch的软件堆栈，以鼓励实验在实验室之间的复制。我们实施了基于强化学习和启发式（基于模型）基线，并展示了从模拟到真实的转移证据。我们的基线在现实世界中实现了20%的成功率；我们的实验确定了未来研究工作提高性能的方法。请访问我们网站上的视频：https://ovmm.github.io/。

English

HomeRobot (noun): An affordable compliant robot that navigates homes and manipulates a wide range of objects in order to complete everyday tasks. Open-Vocabulary Mobile Manipulation (OVMM) is the problem of picking any object in any unseen environment, and placing it in a commanded location. This is a foundational challenge for robots to be useful assistants in human environments, because it involves tackling sub-problems from across robotics: perception, language understanding, navigation, and manipulation are all essential to OVMM. In addition, integration of the solutions to these sub-problems poses its own substantial challenges. To drive research in this area, we introduce the HomeRobot OVMM benchmark, where an agent navigates household environments to grasp novel objects and place them on target receptacles. HomeRobot has two components: a simulation component, which uses a large and diverse curated object set in new, high-quality multi-room home environments; and a real-world component, providing a software stack for the low-cost Hello Robot Stretch to encourage replication of real-world experiments across labs. We implement both reinforcement learning and heuristic (model-based) baselines and show evidence of sim-to-real transfer. Our baselines achieve a 20% success rate in the real world; our experiments identify ways future research work improve performance. See videos on our website: https://ovmm.github.io/.