反思：总结机器人经验以解释和纠正失败

REFLECT: Summarizing Robot Experiences for Failure Explanation and Correction

June 27, 2023

作者: Zeyi Liu, Arpit Bahety, Shuran Song

cs.AI

摘要

自动检测和分析失败执行的能力对于一个可解释且健壮的机器人系统至关重要。最近，大型语言模型（LLMs）已经展示了在文本输入上强大的常识推理能力。为了利用LLM的能力来解释机器人的故障，我们提出了一个名为REFLECT的框架，它将多感官数据转换为机器人过去经验的分层摘要，并使用渐进式故障解释算法向LLM提出查询。根据解释，一个故障修正规划器生成一个可执行计划，使机器人纠正故障并完成任务。为了系统评估这个框架，我们创建了RoboFail数据集，并展示了我们基于LLM的框架能够生成有助于成功修正规划的信息性故障解释。项目网站：https://roboreflect.github.io/

English

The ability to detect and analyze failed executions automatically is crucial for an explainable and robust robotic system. Recently, Large Language Models (LLMs) have demonstrated strong common sense reasoning skills on textual inputs. To leverage the power of LLM for robot failure explanation, we propose a framework REFLECT, which converts multi-sensory data into a hierarchical summary of robot past experiences and queries LLM with a progressive failure explanation algorithm. Conditioned on the explanation, a failure correction planner generates an executable plan for the robot to correct the failure and complete the task. To systematically evaluate the framework, we create the RoboFail dataset and show that our LLM-based framework is able to generate informative failure explanations that assist successful correction planning. Project website: https://roboreflect.github.io/