Habitat 3.0：人类、化身和机器人的共生空间

摘要

我们介绍 Habitat 3.0：一个用于研究家庭环境中协作人机任务的仿真平台。Habitat 3.0 在三个方面做出了贡献：(1) 准确的人形仿真：解决了对复杂可变形体和外观动作多样性进行建模的挑战，同时确保高仿真速度。(2) 人在环回基础设施：通过鼠标/键盘或虚拟现实界面实现真实人类与模拟机器人的交互，促进了通过人类输入评估机器人策略的过程。(3) 协作任务：研究了两个协作任务，社交导航和社交重排。社交导航研究了机器人在未知环境中定位和跟随人形化身的能力，而社交重排则涉及人形和机器人在重排场景时的协作。这些贡献使我们能够深入研究人机协作的端到端学习和启发式基线，并通过人在环评估它们。我们的实验表明，学习的机器人策略在与未知人形代理和可能展示机器人之前未见行为的人类合作时，能够高效完成任务。此外，我们观察到在协作任务执行过程中出现的新行为，比如当机器人阻碍人形代理时，机器人会让出空间，从而使人形代理有效地完成任务。此外，我们使用人在环工具进行的实验表明，通过与真实人类合作者一起评估，我们的自动化评估可以提供不同策略的相对排序指示。Habitat 3.0 在具身人工智能模拟器中开启了有趣的新功能，我们希望它为具身人类-人工智能交互能力的新领域铺平道路。

English

We present Habitat 3.0: a simulation platform for studying collaborative human-robot tasks in home environments. Habitat 3.0 offers contributions across three dimensions: (1) Accurate humanoid simulation: addressing challenges in modeling complex deformable bodies and diversity in appearance and motion, all while ensuring high simulation speed. (2) Human-in-the-loop infrastructure: enabling real human interaction with simulated robots via mouse/keyboard or a VR interface, facilitating evaluation of robot policies with human input. (3) Collaborative tasks: studying two collaborative tasks, Social Navigation and Social Rearrangement. Social Navigation investigates a robot's ability to locate and follow humanoid avatars in unseen environments, whereas Social Rearrangement addresses collaboration between a humanoid and robot while rearranging a scene. These contributions allow us to study end-to-end learned and heuristic baselines for human-robot collaboration in-depth, as well as evaluate them with humans in the loop. Our experiments demonstrate that learned robot policies lead to efficient task completion when collaborating with unseen humanoid agents and human partners that might exhibit behaviors that the robot has not seen before. Additionally, we observe emergent behaviors during collaborative task execution, such as the robot yielding space when obstructing a humanoid agent, thereby allowing the effective completion of the task by the humanoid agent. Furthermore, our experiments using the human-in-the-loop tool demonstrate that our automated evaluation with humanoids can provide an indication of the relative ordering of different policies when evaluated with real human collaborators. Habitat 3.0 unlocks interesting new features in simulators for Embodied AI, and we hope it paves the way for a new frontier of embodied human-AI interaction capabilities.

Habitat 3.0：人类、化身和机器人的共生空间

Habitat 3.0: A Co-Habitat for Humans, Avatars and Robots

摘要

Support