空間記憶智能體：以經驗為基礎的空間智能程序記憶

摘要

空間智能正逐漸成為具身智能體、機器人規劃與多模態助手的基礎能力。為提升VLM智能體的空間推理能力，現有工作主要沿兩條路線展開：一條採用後訓練方法，例如監督式微調與強化學習；另一條則採用智能體範式，使模型呼叫深度估計、3D重建等外部空間工具，以收集中間空間證據。我們研究了一條互補且尚未被充分探索的途徑：凍結的VLM智能體能否在推理時不依賴外部專家空間工具，透過免參數更新的自我演化來提升其空間推理能力？我們提出空間記憶智能體（Spatial Memory Agent, SMA），這是一個基於經驗的運行時框架，可將驗證過的空間經驗轉化為可重複使用、可遷移的經驗教訓。在可驗證的空間環境中，SMA查詢凍結的VLM，取得預測答案與獎勵，並利用驗證器引導的反思，從空間經驗中提煉出簡潔且可遷移的經驗教訓。SMA進一步為每條經驗教訓分配一個遷移可靠性評分（Transfer Reliability Score, TRS），該評分初始為均勻分佈，並根據後續檢索結果進行校準，作為未來遷移可靠性的訪問證據。在唯讀部署階段，SMA透過語義過濾器與相似度-TRS結合排序來檢索經驗教訓，使檢索到的記憶能夠引導凍結模型的推論。在五個具代表性的空間基準與四種基礎VLM上，SMA在每個基礎模型區塊中均達成最高的巨集平均，並在20項評估中的多數情況下取得所評估方法中最高的準確率，從而在所評估的凍結模型規模與環境中，為空間自我演化建立了一條實用的免參數更新路徑。

English

Spatial intelligence is becoming a foundation for embodied agents, robotic planning, and multimodal assistants. To improve the spatial reasoning ability of VLM agents, existing work has mainly followed two lines. One line uses post-training methods, such as supervised fine-tuning and reinforcement learning. Another line adopts an agentic paradigm in which the model calls external spatial tools, such as depth estimation and 3D reconstruction tools, to gather intermediate spatial evidence. We study a complementary and underexplored route: Can a frozen VLM agent improve its spatial reasoning through parameter-update-free self-evolution, without depending on external expert spatial tools at inference time? We present Spatial Memory Agent (SMA), an experience-grounded runtime framework that converts verified spatial experience into reusable transferable lessons. In a verifiable spatial environment, SMA queries the frozen VLM, obtains a predicted answer and reward, and uses verifier-guided reflection to distill compact transferable lessons from spatial experience. SMA further assigns each lesson a Transfer Reliability Score (TRS), which is initialized uniformly and calibrated from later retrieval outcomes as visit evidence of future transfer reliability. During read-only deployment, SMA retrieves lessons by semantic filter and similarity-TRS combined ranking, allowing the retrieved memory to guide frozen model inference. Across five representative spatial benchmarks and four base VLMs, SMA achieves the highest macro average in every base-model block and the best accuracy among the evaluated methods in most of the 20 evaluations, establishing a practical parameter-update-free path for spatial self-evolution across the evaluated frozen model scales and environments.