ESI-Bench: 迈向闭合感知-动作回路的具身空间智能

摘要

空间智能通过感知-行动循环展开：智能体通过行动获取观测数据，并推理观测如何随行动而变化。它们并非被动处理所见之物，而是主动揭示不可见的结构——那些仅凭被动感知无法解析的遮挡结构、动态过程、包含关系及功能属性。我们超越先前假设存在先知观测的空间智能理论框架，将观察者重塑为行动者。我们提出ESI-BENCH——基于OmniGibson构建、扎根于Spelke核心知识系统的综合具身空间智能基准测试，涵盖10个任务类别与29个子类别。智能体需自主决定调用何种能力（感知、移动与操作），并规划执行顺序以主动积累任务相关证据。我们对当前最先进的多模态大语言模型展开广泛实验，发现主动探索显著优于被动方法：智能体无需明确指令便能自发涌现出新兴空间策略，而随机多视角方法反而引入噪声而非有效信息，尽管消耗了更多图像。多数失败并非源于感知薄弱，而是由"行动盲视"所致：不当的行动选择导致低质量观测，进而引发级联错误。尽管显式3D空间锚定能稳定深度敏感任务的推理过程，但非完美的3D表征会扭曲空间关系，其危害程度甚至超过纯2D基线方法。人类对比研究进一步揭示：人类会主动寻找证伪视角并在矛盾中修正信念，而模型无论证据质量如何，均以高置信度过早做出判断——这种元认知鸿沟既无法通过提升感知能力单独弥合，也无法仅通过增强具身交互来消除。

English

Spatial intelligence unfolds through a perception-action loop: agents act to acquire observations, and reason about how observations vary as a function of action. Rather than passively processing what is seen, they actively uncover what is unseen - occluded structure, dynamics, containment, and functionality that cannot be resolved from passive sensing alone. We move beyond prior formulations of spatial intelligence that assume oracle observations by recasting the observer as an actor. We introduce ESI-BENCH, a comprehensive benchmark for embodied spatial intelligence spanning 10 task categories and 29 subcategories built on OmniGibson, grounded in Spelke's core knowledge systems. Agents must decide what abilities to deploy - perception, locomotion, and manipulation - and how to sequence them to actively accumulate task-relevant evidence. We conduct extensive experiments on state-of-the-art MLLMs and find that active exploration substantially outperforms passive counterparts, with agents spontaneously discovering emergent spatial strategies without explicit instructions, while random multi-view often adds noise rather than signal despite consuming far more images. Most failures stem not from weak perception but from action blindness: poor action choices lead to poor observations, which in turn drive cascading errors. While explicit 3D grounding stabilizes reasoning on depth-sensitive tasks, imperfect 3D representation proves more harmful than 2D baselines by distorting spatial relations. Human studies further reveal that unlike humans who seek falsifying viewpoints and revise beliefs under contradiction, models commit prematurely with high confidence regardless of evidence quality, exposing a metacognitive gap that neither better perception nor more embodied interaction alone can close.