双THOR：面向应急感知规划的双臂人形机器人仿真平台

摘要

开发能够在现实场景中执行复杂交互任务的具身智能体，仍然是具身人工智能领域的一项根本性挑战。尽管仿真平台的最新进展极大地提升了训练具身视觉语言模型（VLMs）的任务多样性，但大多数平台依赖于简化的机器人形态，并绕过了底层执行中的随机性，这限制了它们向现实世界机器人的可迁移性。为解决这些问题，我们提出了基于物理的仿真平台DualTHOR，专为复杂双臂人形机器人设计，该平台建立在AI2-THOR的扩展版本之上。我们的模拟器包含真实世界机器人资产、一套双臂协作任务集以及人形机器人的逆运动学求解器。此外，我们还引入了一种应急机制，通过基于物理的底层执行来模拟潜在故障，从而缩小与现实场景的差距。我们的模拟器使得在家庭环境中对VLMs的鲁棒性和泛化能力进行更全面的评估成为可能。大量评估表明，当前的VLMs在双臂协调方面存在困难，并且在包含应急情况的现实环境中表现出有限的鲁棒性，这凸显了使用我们的模拟器来开发更具能力的VLMs以执行具身任务的重要性。代码已发布于https://github.com/ds199895/DualTHOR.git。

English

Developing embodied agents capable of performing complex interactive tasks in real-world scenarios remains a fundamental challenge in embodied AI. Although recent advances in simulation platforms have greatly enhanced task diversity to train embodied Vision Language Models (VLMs), most platforms rely on simplified robot morphologies and bypass the stochastic nature of low-level execution, which limits their transferability to real-world robots. To address these issues, we present a physics-based simulation platform DualTHOR for complex dual-arm humanoid robots, built upon an extended version of AI2-THOR. Our simulator includes real-world robot assets, a task suite for dual-arm collaboration, and inverse kinematics solvers for humanoid robots. We also introduce a contingency mechanism that incorporates potential failures through physics-based low-level execution, bridging the gap to real-world scenarios. Our simulator enables a more comprehensive evaluation of the robustness and generalization of VLMs in household environments. Extensive evaluations reveal that current VLMs struggle with dual-arm coordination and exhibit limited robustness in realistic environments with contingencies, highlighting the importance of using our simulator to develop more capable VLMs for embodied tasks. The code is available at https://github.com/ds199895/DualTHOR.git.

双THOR：面向应急感知规划的双臂人形机器人仿真平台

DualTHOR: A Dual-Arm Humanoid Simulation Platform for Contingency-Aware Planning

摘要

Support