GRUtopia：規模城市中的夢幻通用機器人

GRUtopia: Dream General Robots in a City at Scale

July 15, 2024

作者: Hanqing Wang, Jiahe Chen, Wensi Huang, Qingwei Ben, Tai Wang, Boyu Mi, Tao Huang, Siheng Zhao, Yilun Chen, Sizhe Yang, Peizhou Cao, Wenye Yu, Zichao Ye, Jialun Li, Junfeng Long, Zirui Wang, Huiling Wang, Ying Zhao, Zhongying Tu, Yu Qiao, Dahua Lin, Jiangmiao Pang

cs.AI

摘要

最近的研究一直在探索具體化人工智慧領域中的擴展規律。鑑於收集真實世界數據的高昂成本，我們認為模擬到真實（Sim2Real）範式是擴展具體化模型學習的關鍵一步。本文介紹了GRUtopia項目，這是第一個為各種機器人設計的模擬互動式3D社會。它具有幾個先進之處：(a) 場景數據集GRScenes包括10萬個互動式、精細標註的場景，可以自由組合成城市規模的環境。與先前主要專注於家庭的作品不同，GRScenes涵蓋了89個不同的場景類別，彌合了服務導向環境的鴻溝，這是通用機器人最初可能部署的地方。(b) GRResidents是一個由大型語言模型（LLM）驅動的非玩家角色（NPC）系統，負責社交互動、任務生成和任務分配，從而模擬具體化人工智慧應用的社交場景。(c) 基準測試GRBench支持各種機器人，但主要專注於四肢機器人作為主要代理人，提出涉及物體定位導航、社交導航和定位操作的中等挑戰性任務。我們希望這項工作可以緩解這一領域高質量數據的匱乏問題，並提供對具體化人工智慧研究更全面的評估。該項目可在https://github.com/OpenRobotLab/GRUtopia找到。

English

Recent works have been exploring the scaling laws in the field of Embodied AI. Given the prohibitive costs of collecting real-world data, we believe the Simulation-to-Real (Sim2Real) paradigm is a crucial step for scaling the learning of embodied models. This paper introduces project GRUtopia, the first simulated interactive 3D society designed for various robots. It features several advancements: (a) The scene dataset, GRScenes, includes 100k interactive, finely annotated scenes, which can be freely combined into city-scale environments. In contrast to previous works mainly focusing on home, GRScenes covers 89 diverse scene categories, bridging the gap of service-oriented environments where general robots would be initially deployed. (b) GRResidents, a Large Language Model (LLM) driven Non-Player Character (NPC) system that is responsible for social interaction, task generation, and task assignment, thus simulating social scenarios for embodied AI applications. (c) The benchmark, GRBench, supports various robots but focuses on legged robots as primary agents and poses moderately challenging tasks involving Object Loco-Navigation, Social Loco-Navigation, and Loco-Manipulation. We hope that this work can alleviate the scarcity of high-quality data in this field and provide a more comprehensive assessment of Embodied AI research. The project is available at https://github.com/OpenRobotLab/GRUtopia.