ChatPaper.aiChatPaper

大型語言模型下的創意機器人工具使用

Creative Robot Tool Use with Large Language Models

October 19, 2023
作者: Mengdi Xu, Peide Huang, Wenhao Yu, Shiqi Liu, Xilun Zhang, Yaru Niu, Tingnan Zhang, Fei Xia, Jie Tan, Ding Zhao
cs.AI

摘要

工具使用是高級智能的標誌,無論是在動物行為還是機器人能力中都有體現。本文探討了賦予機器人創造性地使用工具的能力,以應對涉及隱含物理限制和長期規劃的任務的可行性。利用大型語言模型(LLMs),我們開發了RoboTool,這是一個系統,可以接受自然語言指令並輸出控制機器人在模擬和現實環境中的可執行代碼。RoboTool包含四個關鍵組件:(i)一個“分析器”,用於解釋自然語言以識別關鍵任務相關概念,(ii)一個“規劃器”,根據語言輸入和關鍵概念生成全面策略,(iii)一個“計算器”,用於計算每個技能的參數,以及(iv)一個“編碼器”,將這些計劃轉換為可執行的Python代碼。我們的結果表明,RoboTool不僅可以理解明確或隱含的物理限制和環境因素,還可以展示創造性的工具使用。與傳統的任務和運動規劃(TAMP)方法依賴於明確優化不同,我們基於LLM的系統為複雜的機器人任務提供了更靈活、高效和用戶友好的解決方案。通過大量實驗,我們驗證了RoboTool在處理否則無法實現的任務方面表現出色,這擴展了機器人系統的能力。我們的項目頁面提供了演示:https://creative-robotool.github.io/。
English
Tool use is a hallmark of advanced intelligence, exemplified in both animal behavior and robotic capabilities. This paper investigates the feasibility of imbuing robots with the ability to creatively use tools in tasks that involve implicit physical constraints and long-term planning. Leveraging Large Language Models (LLMs), we develop RoboTool, a system that accepts natural language instructions and outputs executable code for controlling robots in both simulated and real-world environments. RoboTool incorporates four pivotal components: (i) an "Analyzer" that interprets natural language to discern key task-related concepts, (ii) a "Planner" that generates comprehensive strategies based on the language input and key concepts, (iii) a "Calculator" that computes parameters for each skill, and (iv) a "Coder" that translates these plans into executable Python code. Our results show that RoboTool can not only comprehend explicit or implicit physical constraints and environmental factors but also demonstrate creative tool use. Unlike traditional Task and Motion Planning (TAMP) methods that rely on explicit optimization, our LLM-based system offers a more flexible, efficient, and user-friendly solution for complex robotics tasks. Through extensive experiments, we validate that RoboTool is proficient in handling tasks that would otherwise be infeasible without the creative use of tools, thereby expanding the capabilities of robotic systems. Demos are available on our project page: https://creative-robotool.github.io/.
PDF91December 15, 2024