ChatPaper.aiChatPaper

将基础模型构建为物理世界智能体,推进长时程导航前沿

Scaffolding Foundation Models into Physical-World Agents Pushes the Frontier of Long-Horizon Navigation

August 31, 2026
作者: Zixing Lei, Gengze Zhou, Xiong-Hui Chen, Jiazhao Zhang, Yiyang Huang, Hang Yin, Haoqi Yuan, Qi Wu, Weixin Li, Siheng Chen
cs.AI

摘要

长时程物理世界智能体必须在推理远期目标的同时,将决策建立在可靠的闭环行为之上。当前的基础模型将这两种能力割裂开来:视觉-语言模型(VLM)能够推断缺失信息并调整高层规划,但在重复的导航落地中仍然脆弱且低效;而导航基础模型(NFM)虽能稳健地执行语义目标,却只能在有界回合中运行,缺乏持久的任务级推理。我们提出了 NavMCP,一个智能体脚手架框架,将 VLM 推理智能体与 NFM 执行器耦合,以实现长时程探索。VLM 决定寻找什么证据、在哪里搜索以及何时停止,而 NFM 则将每个语义子目标落地为闭环导航。三个通道构成了它们的协作:意图通道将证据需求转化为导航调用,观测通道将轨迹展开转换为来源可溯的轨迹证据,记忆通道则跨调用累积发现、负面证据和未解决的目标。这种设计将孤立的导航轨迹展开转变为持久的具身交互,无需对任一模型进行重新训练。在具身问答任务上,NavMCP 在 HM-EQA、MT-HM3D 和 EXPRESS-Bench 上取得了最先进的结果。在匹配智能体与执行器主干网络的情况下,它在 HM-EQA 上比回合制接口高出 14.9 个百分点。在宇树 Go2 机器人上,NavMCP 达到了 78.3% 的成功率,随着任务时程增长,其相对于最强基线的优势从 10 个百分点扩大到 45 个百分点。这些结果展示了将互补的基础模型通过脚手架方式整合为长时程物理世界智能体的潜力。
English
Long-horizon physical-world agents must reason over distant goals while grounding decisions in reliable closed-loop behavior. Today's foundation models split these capabilities: vision-language models (VLMs) infer missing information and adapt high-level plans but remain brittle and inefficient at repeated navigation grounding, while navigation foundation models (NFMs) robustly execute semantic goals but operate as bounded episodes without persistent task-level reasoning. We introduce NavMCP, an agentic scaffolding framework that couples a VLM reasoning agent with an NFM executor for long-horizon exploration. The VLM decides what evidence to seek, where to search, and when to stop, while the NFM grounds each semantic sub-goal into closed-loop navigation. Three channels structure their collaboration: intent translates evidence needs into navigation calls, observation converts rollouts into source-grounded trajectory evidence, and memory accumulates findings, negative evidence, and unresolved goals across calls. This design turns isolated navigation rollouts into persistent embodied interaction without retraining either model. On Embodied Question Answering, NavMCP achieves state-of-the-art results on HM-EQA, MT-HM3D, and EXPRESS-Bench. Under matched agent and executor backbones, it outperforms an episodic interface by 14.9 percentage points on HM-EQA. On a Unitree Go2, NavMCP reaches 78.3% success, with its margin over the strongest baseline growing from 10 to 45 points as the task horizon increases. These results demonstrate the potential of scaffolding complementary foundation models into long-horizon physical-world agents.