HomeRobot：開放詞彙的移動操作

HomeRobot: Open-Vocabulary Mobile Manipulation

June 20, 2023

作者: Sriram Yenamandra, Arun Ramachandran, Karmesh Yadav, Austin Wang, Mukul Khanna, Theophile Gervet, Tsung-Yen Yang, Vidhi Jain, Alexander William Clegg, John Turner, Zsolt Kira, Manolis Savva, Angel Chang, Devendra Singh Chaplot, Dhruv Batra, Roozbeh Mottaghi, Yonatan Bisk, Chris Paxton

cs.AI

摘要

HomeRobot（名詞）：一款價格親民的遵從性機器人，能夠在家中導航並操作各種物件，以完成日常任務。開放詞彙移動操作（OVMM）是在任何未知環境中拾取任何物件並將其放置在指定位置的問題。這是機器人成為人類環境中有用助手的基礎挑戰，因為它涉及解決機器人各個領域的子問題：感知、語言理解、導航和操作對於OVMM都是至關重要的。此外，整合這些子問題的解決方案本身也具有重大挑戰。為了推動這一領域的研究，我們引入了HomeRobot OVMM基準測試，其中一個代理在家庭環境中導航，抓取新穎物件並將其放置在目標容器中。HomeRobot有兩個組件：一個模擬組件，使用新的高質量多房間家庭環境中的大量多樣化物件集合；以及一個現實世界組件，提供一個針對低成本Hello Robot Stretch的軟體堆疊，以鼓勵實驗室之間進行現實世界實驗的複製。我們實施了基於強化學習和啟發式（基於模型）的基準線，並展示了從模擬到現實的轉移證據。我們的基準線在現實世界實現了20%的成功率；我們的實驗確定了未來研究工作可以改善性能的方法。在我們的網站上觀看視頻：https://ovmm.github.io/。

English

HomeRobot (noun): An affordable compliant robot that navigates homes and manipulates a wide range of objects in order to complete everyday tasks. Open-Vocabulary Mobile Manipulation (OVMM) is the problem of picking any object in any unseen environment, and placing it in a commanded location. This is a foundational challenge for robots to be useful assistants in human environments, because it involves tackling sub-problems from across robotics: perception, language understanding, navigation, and manipulation are all essential to OVMM. In addition, integration of the solutions to these sub-problems poses its own substantial challenges. To drive research in this area, we introduce the HomeRobot OVMM benchmark, where an agent navigates household environments to grasp novel objects and place them on target receptacles. HomeRobot has two components: a simulation component, which uses a large and diverse curated object set in new, high-quality multi-room home environments; and a real-world component, providing a software stack for the low-cost Hello Robot Stretch to encourage replication of real-world experiments across labs. We implement both reinforcement learning and heuristic (model-based) baselines and show evidence of sim-to-real transfer. Our baselines achieve a 20% success rate in the real world; our experiments identify ways future research work improve performance. See videos on our website: https://ovmm.github.io/.