GOAT：GO to Any Thing

GOAT: GO to Any Thing

November 10, 2023

作者: Matthew Chang, Theophile Gervet, Mukul Khanna, Sriram Yenamandra, Dhruv Shah, So Yeon Min, Kavit Shah, Chris Paxton, Saurabh Gupta, Dhruv Batra, Roozbeh Mottaghi, Jitendra Malik, Devendra Singh Chaplot

cs.AI

摘要

在家庭和倉庫等部署場景中，移動機器人被期望能夠自主導航長時間，無縫地執行由人類操作者直觀理解的任務。我們提出了 GO To Any Thing (GOAT)，這是一個通用導航系統，具有三個關鍵特徵：a) 多模態：可以處理通過類別標籤、目標圖像和語言描述指定的目標，b) 終身學習：受益於在相同環境中的過去經驗，以及c) 平台無關：可以快速部署在具有不同實體的機器人上。GOAT通過模塊化系統設計和不斷擴充的實例感知語義記憶實現，該記憶不僅跟踪物體的外觀，還包括不同視角下的物體外觀，以及類別級別的語義。這使得GOAT能夠區分同一類別的不同實例，以實現導航到通過圖像和語言描述指定的目標。在實驗比較中，我們在9個不同家庭的超過90小時的時間內，選擇了675個目標，包括200多個不同的物體實例，發現GOAT實現了83%的整體成功率，比之前的方法和消融方法提高了32%（絕對改善）。GOAT在環境中的經驗中不斷改進，從第一個目標的60%成功率到探索後的90%成功率。此外，我們展示了GOAT可以輕鬆應用於拾取和放置以及社交導航等下游任務。

English

In deployment scenarios such as homes and warehouses, mobile robots are expected to autonomously navigate for extended periods, seamlessly executing tasks articulated in terms that are intuitively understandable by human operators. We present GO To Any Thing (GOAT), a universal navigation system capable of tackling these requirements with three key features: a) Multimodal: it can tackle goals specified via category labels, target images, and language descriptions, b) Lifelong: it benefits from its past experience in the same environment, and c) Platform Agnostic: it can be quickly deployed on robots with different embodiments. GOAT is made possible through a modular system design and a continually augmented instance-aware semantic memory that keeps track of the appearance of objects from different viewpoints in addition to category-level semantics. This enables GOAT to distinguish between different instances of the same category to enable navigation to targets specified by images and language descriptions. In experimental comparisons spanning over 90 hours in 9 different homes consisting of 675 goals selected across 200+ different object instances, we find GOAT achieves an overall success rate of 83%, surpassing previous methods and ablations by 32% (absolute improvement). GOAT improves with experience in the environment, from a 60% success rate at the first goal to a 90% success after exploration. In addition, we demonstrate that GOAT can readily be applied to downstream tasks such as pick and place and social navigation.