ChatPaper.aiChatPaper

將基礎模型支架化為實體世界智能體,拓展了長程導航的前沿

Scaffolding Foundation Models into Physical-World Agents Pushes the Frontier of Long-Horizon Navigation

August 31, 2026
作者: Zixing Lei, Gengze Zhou, Xiong-Hui Chen, Jiazhao Zhang, Yiyang Huang, Hang Yin, Haoqi Yuan, Qi Wu, Weixin Li, Siheng Chen
cs.AI

摘要

長期視野的物理世界代理人必須在依賴可靠的閉環行為來落實決策的同時,對遙遠目標進行推理。當今的基礎模型將這些能力分割開來:視覺語言模型(VLM)能推斷缺失資訊並調整高層級計畫,但在重複的導航落地方面仍顯脆弱且低效;而導航基礎模型(NFM)能穩健地執行語意目標,但僅限於有邊界的回合,缺乏持續性的任務層級推理。我們提出 NavMCP,一個代理性脚手架框架,將 VLM 推理代理與 NFM 執行器結合,以進行長期視野的探索。VLM 決定要尋找哪些證據、往何處搜尋,以及何時停止;而 NFM 則將每個語意子目標落地為閉環導航。三個通道構成它們的協作架構:意圖(intent)將證據需求轉換為導航呼叫;觀測(observation)將軌跡轉換為具有來源根據的軌跡證據;記憶(memory)則跨呼叫累積發現、負面證據與未解決目標。此設計將孤立的導航軌跡轉化為持續的具身互動,且無需重新訓練任一模型。在具身問答任務上,NavMCP 在 HM-EQA、MT-HM3D 與 EXPRESS-Bench 達到當前最佳結果。在配對代理與執行器骨幹的條件下,它在 HM-EQA 上比回合制介面高出 14.9 個百分點。在 Unitree Go2 上,NavMCP 達到 78.3% 的成功率,且其相較於最強基線的領先幅度隨著任務視野拉長而從 10 個百分點成長至 45 個百分點。這些結果展現了將互補的基礎模型組裝為長期視野物理世界代理人的潛力。
English
Long-horizon physical-world agents must reason over distant goals while grounding decisions in reliable closed-loop behavior. Today's foundation models split these capabilities: vision-language models (VLMs) infer missing information and adapt high-level plans but remain brittle and inefficient at repeated navigation grounding, while navigation foundation models (NFMs) robustly execute semantic goals but operate as bounded episodes without persistent task-level reasoning. We introduce NavMCP, an agentic scaffolding framework that couples a VLM reasoning agent with an NFM executor for long-horizon exploration. The VLM decides what evidence to seek, where to search, and when to stop, while the NFM grounds each semantic sub-goal into closed-loop navigation. Three channels structure their collaboration: intent translates evidence needs into navigation calls, observation converts rollouts into source-grounded trajectory evidence, and memory accumulates findings, negative evidence, and unresolved goals across calls. This design turns isolated navigation rollouts into persistent embodied interaction without retraining either model. On Embodied Question Answering, NavMCP achieves state-of-the-art results on HM-EQA, MT-HM3D, and EXPRESS-Bench. Under matched agent and executor backbones, it outperforms an episodic interface by 14.9 percentage points on HM-EQA. On a Unitree Go2, NavMCP reaches 78.3% success, with its margin over the strongest baseline growing from 10 to 45 points as the task horizon increases. These results demonstrate the potential of scaffolding complementary foundation models into long-horizon physical-world agents.