空间记忆智能体：经验锚定的程序性记忆用于空间智能

摘要

空间智能正在成为具身智能体、机器人规划和多模态助手的基础。为提升VLM智能体的空间推理能力，现有工作主要沿两条路线展开：一条路线采用后训练方法，如监督微调和强化学习；另一条路线采用智能体范式，模型调用外部空间工具（如深度估计和三维重建工具）以收集中间空间证据。我们研究了一条互补且尚未被充分探索的路径：冻结的VLM智能体能否在不依赖外部专家空间工具推理的情况下，通过无参数更新的自我进化来提升其空间推理能力？我们提出空间记忆智能体（Spatial Memory Agent, SMA），一种基于经验的运行时框架，将经验证的空间经验转化为可复用的可迁移经验。在可验证的空间环境中，SMA查询冻结的VLM，获取预测答案和奖励，并利用验证器引导的反思从空间经验中提炼紧凑的可迁移经验。SMA进一步为每条经验分配一个迁移可靠性评分（Transfer Reliability Score, TRS），该评分初始化为均匀分布，并根据后续检索结果进行校准，作为未来迁移可靠性的访问证据。在只读部署阶段，SMA通过语义过滤器和相似度-TRS联合排序来检索经验，使检索到的记忆能够引导冻结模型的推理。在五个代表性空间基准和四个基础VLM上，SMA在每组基础模型上均取得最高宏平均准确率，并在20项评估中的大部分中取得最佳准确率，为所评估的冻结模型规模和环境中的空间自我进化建立了一条实用的无参数更新路径。

English

Spatial intelligence is becoming a foundation for embodied agents, robotic planning, and multimodal assistants. To improve the spatial reasoning ability of VLM agents, existing work has mainly followed two lines. One line uses post-training methods, such as supervised fine-tuning and reinforcement learning. Another line adopts an agentic paradigm in which the model calls external spatial tools, such as depth estimation and 3D reconstruction tools, to gather intermediate spatial evidence. We study a complementary and underexplored route: Can a frozen VLM agent improve its spatial reasoning through parameter-update-free self-evolution, without depending on external expert spatial tools at inference time? We present Spatial Memory Agent (SMA), an experience-grounded runtime framework that converts verified spatial experience into reusable transferable lessons. In a verifiable spatial environment, SMA queries the frozen VLM, obtains a predicted answer and reward, and uses verifier-guided reflection to distill compact transferable lessons from spatial experience. SMA further assigns each lesson a Transfer Reliability Score (TRS), which is initialized uniformly and calibrated from later retrieval outcomes as visit evidence of future transfer reliability. During read-only deployment, SMA retrieves lessons by semantic filter and similarity-TRS combined ranking, allowing the retrieved memory to guide frozen model inference. Across five representative spatial benchmarks and four base VLMs, SMA achieves the highest macro average in every base-model block and the best accuracy among the evaluated methods in most of the 20 evaluations, establishing a practical parameter-update-free path for spatial self-evolution across the evaluated frozen model scales and environments.