SpatialCLI：學習使用空間工具進行推理，然後在無工具情況下推理

摘要

視覺-語言模型（VLMs）日益被用於具身智能體中，以解釋視覺輸入、推理空間關係，並基於該推理做出任務層級的決策。然而，一個根本性的能力錯配仍然存在：通用視覺-語言模型能夠推理整體任務，但往往遺漏決定成敗的視覺細節；而專業視覺模型雖然能捕捉這些細節，卻無法將其轉化為任務層級的決策。在本工作中，我們提出 SpatialCLI，一個教導視覺-語言模型使用空間工具進行推理，並逐步內化這些工具所提供之專業感知能力的框架。SpatialCLI 分三個階段進行：（1）調用（Call）將專業視覺模型暴露為空間工具，以增強視覺-語言模型的感知能力；（2）學習（Learn）利用冷啟動監督微調（Cold-Start SFT）和智能體強化學習（agentic RL）來改善工具使用；（3）內化（Internalize）將成功的工具使用軌跡語言化，以內化專業感知能力。我們進一步引入 SpatialCLI-Bench，一個包含 516 個範例的組合式感知基準，涵蓋定位、分割、深度和姿態。在 MindCube 上，SpatialCLI 使 Qwen3-VL-8B-Instruct 從 29.3% 提升至搭配工具時的 84.6%，超越了搭配工具時的 GPT-5.6 Sol（72.1%），同時在內化後不搭配工具的情況下仍保持 73.8% 的表現。

English

Vision-language models (VLMs) are increasingly used in embodied agents to interpret visual inputs, reason about spatial relationships, and make task-level decisions based on that reasoning. However, a fundamental capability mismatch remains: general VLMs can reason about the overall task but often miss the visual details that determine success, while specialist vision models can capture those details but cannot translate them into task-level decisions. In this work, we propose SpatialCLI, a framework that teaches VLMs to reason with spatial tools and progressively internalize the specialist perceptual capabilities they provide. SpatialCLI proceeds in three stages: (1) Call exposes specialist vision models as spatial tools to augment the VLM's perception; (2) Learn uses Cold-Start SFT and agentic RL to improve tool use; and (3) Internalize verbalizes successful tool-use trajectories to internalize specialist perceptual capabilities. We further introduce SpatialCLI-Bench, a 516-example benchmark for compositional perception across localization, segmentation, depth, and pose. On MindCube, SpatialCLI raises Qwen3-VL-8B-Instruct from 29.3% to 84.6% with tools, surpassing GPT-5.6 Sol with tools (72.1%), while retaining 73.8% without tools after internalization.