GOAT：GO to Any Thing

GOAT: GO to Any Thing

November 10, 2023

作者: Matthew Chang, Theophile Gervet, Mukul Khanna, Sriram Yenamandra, Dhruv Shah, So Yeon Min, Kavit Shah, Chris Paxton, Saurabh Gupta, Dhruv Batra, Roozbeh Mottaghi, Jitendra Malik, Devendra Singh Chaplot

cs.AI

摘要

在家庭和仓库等部署场景中，移动机器人被期望能够自主导航长时间，无缝执行任务，这些任务以人类操作者直观理解的方式表达。我们提出了GO To Any Thing（GOAT），这是一个通用导航系统，具有三个关键特性：a）多模态：它可以处理通过类别标签、目标图像和语言描述指定的目标，b）终身学习：它从过去在相同环境中的经验中受益，c）平台无关：它可以快速部署在具有不同实体的机器人上。GOAT通过模块化系统设计和不断增强的实例感知语义记忆实现，该记忆跟踪不同视角中对象的外观，除了类别级语义。这使GOAT能够区分同一类别的不同实例，以便导航到由图像和语言描述指定的目标。在实验比较中，跨越了9个不同家庭的90多个小时，包括200多个不同对象实例的675个目标，我们发现GOAT实现了83%的总体成功率，超过了以前的方法和消融实验32%（绝对改善）。GOAT在环境中的经验增加后表现得更好，从第一个目标的60%成功率到探索后的90%成功率。此外，我们展示了GOAT可以轻松应用于拾取放置和社交导航等下游任务。

English

In deployment scenarios such as homes and warehouses, mobile robots are expected to autonomously navigate for extended periods, seamlessly executing tasks articulated in terms that are intuitively understandable by human operators. We present GO To Any Thing (GOAT), a universal navigation system capable of tackling these requirements with three key features: a) Multimodal: it can tackle goals specified via category labels, target images, and language descriptions, b) Lifelong: it benefits from its past experience in the same environment, and c) Platform Agnostic: it can be quickly deployed on robots with different embodiments. GOAT is made possible through a modular system design and a continually augmented instance-aware semantic memory that keeps track of the appearance of objects from different viewpoints in addition to category-level semantics. This enables GOAT to distinguish between different instances of the same category to enable navigation to targets specified by images and language descriptions. In experimental comparisons spanning over 90 hours in 9 different homes consisting of 675 goals selected across 200+ different object instances, we find GOAT achieves an overall success rate of 83%, surpassing previous methods and ablations by 32% (absolute improvement). GOAT improves with experience in the environment, from a 60% success rate at the first goal to a 90% success after exploration. In addition, we demonstrate that GOAT can readily be applied to downstream tasks such as pick and place and social navigation.