CANVAS：基于常识的导航系统，用于直观的人机交互

CANVAS: Commonsense-Aware Navigation System for Intuitive Human-Robot Interaction

October 2, 2024

作者: Suhwan Choi, Yongjun Cho, Minchan Kim, Jaeyoon Jung, Myunchul Joe, Yubeen Park, Minseo Kim, Sungwoong Kim, Sungjae Lee, Hwiseong Park, Jiwan Chung, Youngjae Yu

cs.AI

摘要

现实生活中的机器人导航不仅仅涉及到达目的地；它需要在解决特定场景目标的同时优化移动。人类表达这些目标的直观方式是通过抽象线索，如口头命令或粗略草图。这种人类引导可能缺乏细节或带有噪音。尽管如此，我们期望机器人按照预期进行导航。为了使机器人能够理解并执行这些抽象指令，符合人类期望，它们必须与人类共享基本导航概念的共同理解。为此，我们引入了CANVAS，这是一个结合视觉和语言指令的常识感知导航新框架。它的成功源于模仿学习，使机器人能够从人类导航行为中学习。我们提出了COMMAND，这是一个包含人类注释的导航结果的全面数据集，涵盖48小时和219公里，旨在训练在模拟环境中进行常识感知导航的系统。我们的实验表明，CANVAS在所有环境中均优于强大的基于规则的系统ROS NavStack，展现出在嘈杂指令下的卓越性能。值得注意的是，在果园环境中，ROS NavStack记录了0%的总成功率，而CANVAS实现了67%的总成功率。CANVAS还与人类演示和常识约束紧密对齐，即使在未知环境中也是如此。此外，CANVAS的实际部署展示了令人印象深刻的Sim2Real转移，总成功率达到69%，突显了从模拟环境中学习人类演示对实际应用的潜力。

English

Real-life robot navigation involves more than just reaching a destination; it requires optimizing movements while addressing scenario-specific goals. An intuitive way for humans to express these goals is through abstract cues like verbal commands or rough sketches. Such human guidance may lack details or be noisy. Nonetheless, we expect robots to navigate as intended. For robots to interpret and execute these abstract instructions in line with human expectations, they must share a common understanding of basic navigation concepts with humans. To this end, we introduce CANVAS, a novel framework that combines visual and linguistic instructions for commonsense-aware navigation. Its success is driven by imitation learning, enabling the robot to learn from human navigation behavior. We present COMMAND, a comprehensive dataset with human-annotated navigation results, spanning over 48 hours and 219 km, designed to train commonsense-aware navigation systems in simulated environments. Our experiments show that CANVAS outperforms the strong rule-based system ROS NavStack across all environments, demonstrating superior performance with noisy instructions. Notably, in the orchard environment, where ROS NavStack records a 0% total success rate, CANVAS achieves a total success rate of 67%. CANVAS also closely aligns with human demonstrations and commonsense constraints, even in unseen environments. Furthermore, real-world deployment of CANVAS showcases impressive Sim2Real transfer with a total success rate of 69%, highlighting the potential of learning from human demonstrations in simulated environments for real-world applications.