Habitat 3.0：人類、虛擬人和機器人的共同棲息地

摘要

我們介紹 Habitat 3.0：一個用於研究家庭環境中協作人機任務的模擬平台。Habitat 3.0 在三個維度上做出貢獻：(1) 準確的人形模擬：應對在建模複雜可變形物體和外觀運動多樣性方面的挑戰，同時確保高速度模擬。 (2) 人機互動基礎設施：通過滑鼠/鍵盤或虛擬實境界面實現真實人類與模擬機器人的互動，促進通過人類輸入評估機器人策略。 (3) 協作任務：研究兩個協作任務，社交導航和社交重新排列。社交導航探討機器人在未知環境中定位和跟隨人形化身的能力，而社交重新排列則處理人形和機器人在重新排列場景時的協作。這些貢獻使我們能夠深入研究人機協作的端對端學習和啟發式基準，並通過人機互動進行評估。我們的實驗表明，當與未知的人形代理和可能展現機器人未見行為的人類合作時，學習的機器人策略導致有效的任務完成。此外，我們觀察到在協作任務執行過程中出現的新行為，例如當機器人阻礙人形代理時，讓出空間，從而使人形代理有效完成任務。此外，我們使用人機互動工具的實驗表明，我們與人形代理進行的自動評估可以提供對不同策略進行實際人類合作者評估時的相對排序的指示。Habitat 3.0 在具體 AI 模擬器中解鎖了有趣的新功能，我們希望它為具體化的人機互動能力開拓出一個新的前沿。

English

We present Habitat 3.0: a simulation platform for studying collaborative human-robot tasks in home environments. Habitat 3.0 offers contributions across three dimensions: (1) Accurate humanoid simulation: addressing challenges in modeling complex deformable bodies and diversity in appearance and motion, all while ensuring high simulation speed. (2) Human-in-the-loop infrastructure: enabling real human interaction with simulated robots via mouse/keyboard or a VR interface, facilitating evaluation of robot policies with human input. (3) Collaborative tasks: studying two collaborative tasks, Social Navigation and Social Rearrangement. Social Navigation investigates a robot's ability to locate and follow humanoid avatars in unseen environments, whereas Social Rearrangement addresses collaboration between a humanoid and robot while rearranging a scene. These contributions allow us to study end-to-end learned and heuristic baselines for human-robot collaboration in-depth, as well as evaluate them with humans in the loop. Our experiments demonstrate that learned robot policies lead to efficient task completion when collaborating with unseen humanoid agents and human partners that might exhibit behaviors that the robot has not seen before. Additionally, we observe emergent behaviors during collaborative task execution, such as the robot yielding space when obstructing a humanoid agent, thereby allowing the effective completion of the task by the humanoid agent. Furthermore, our experiments using the human-in-the-loop tool demonstrate that our automated evaluation with humanoids can provide an indication of the relative ordering of different policies when evaluated with real human collaborators. Habitat 3.0 unlocks interesting new features in simulators for Embodied AI, and we hope it paves the way for a new frontier of embodied human-AI interaction capabilities.

Habitat 3.0：人類、虛擬人和機器人的共同棲息地

Habitat 3.0: A Co-Habitat for Humans, Avatars and Robots

摘要

Support