GRUtopia：规模城市中的梦幻通用机器人

GRUtopia: Dream General Robots in a City at Scale

July 15, 2024

作者: Hanqing Wang, Jiahe Chen, Wensi Huang, Qingwei Ben, Tai Wang, Boyu Mi, Tao Huang, Siheng Zhao, Yilun Chen, Sizhe Yang, Peizhou Cao, Wenye Yu, Zichao Ye, Jialun Li, Junfeng Long, Zirui Wang, Huiling Wang, Ying Zhao, Zhongying Tu, Yu Qiao, Dahua Lin, Jiangmiao Pang

cs.AI

摘要

最近的研究一直在探索具身体AI领域的规模定律。考虑到收集真实世界数据的成本很高，我们认为“从模拟到真实”（Sim2Real）范式是扩展具身体模型学习的关键一步。本文介绍了GRUtopia项目，这是第一个为各种机器人设计的模拟互动3D社会。它具有几项先进之处：(a) 场景数据集GRScenes包括10万个互动、精细注释的场景，可以自由组合成城市规模的环境。与以往主要关注家庭的作品不同，GRScenes涵盖了89种不同的场景类别，弥合了服务导向环境与普通机器人最初部署的差距。(b) GRResidents是一个由大型语言模型（LLM）驱动的非玩家角色（NPC）系统，负责社交互动、任务生成和任务分配，从而为具身体AI应用模拟社交场景。(c) 基准测试GRBench支持各种机器人，但主要关注四肢机器人作为主要代理，并提出涉及物体定位导航、社交导航和定位操作的适度具有挑战性的任务。我们希望这项工作可以缓解该领域高质量数据的稀缺问题，并提供对具身体AI研究更全面的评估。该项目可在https://github.com/OpenRobotLab/GRUtopia找到。

English

Recent works have been exploring the scaling laws in the field of Embodied AI. Given the prohibitive costs of collecting real-world data, we believe the Simulation-to-Real (Sim2Real) paradigm is a crucial step for scaling the learning of embodied models. This paper introduces project GRUtopia, the first simulated interactive 3D society designed for various robots. It features several advancements: (a) The scene dataset, GRScenes, includes 100k interactive, finely annotated scenes, which can be freely combined into city-scale environments. In contrast to previous works mainly focusing on home, GRScenes covers 89 diverse scene categories, bridging the gap of service-oriented environments where general robots would be initially deployed. (b) GRResidents, a Large Language Model (LLM) driven Non-Player Character (NPC) system that is responsible for social interaction, task generation, and task assignment, thus simulating social scenarios for embodied AI applications. (c) The benchmark, GRBench, supports various robots but focuses on legged robots as primary agents and poses moderately challenging tasks involving Object Loco-Navigation, Social Loco-Navigation, and Loco-Manipulation. We hope that this work can alleviate the scarcity of high-quality data in this field and provide a more comprehensive assessment of Embodied AI research. The project is available at https://github.com/OpenRobotLab/GRUtopia.