当记忆说谎时：VLM智能体空间记忆陈旧性的实证研究

摘要

记忆增强的视觉语言模型智能体依赖持久化空间知识进行决策，然而当环境发生变化时，这些知识会悄然变得陈旧。我们探究的问题是：当智能体必须调和一项自信的记忆断言与一条相互矛盾的观察时会发生什么，以及现有模型能否在冲突演变为与安全相关的错误之前察觉它。基于动态FrozenLake测试平台，我们将陈旧性检测任务与下游导航任务相结合，在三种闭源模型和三种开放权重视觉语言模型上，分别以文本和图像输入进行评测（共1,800次检测运行，以及4个LLM导航器在共享的50个随机种子规模下的12,000次文本模式导航回合）。我们得出三点发现。第一，文本可解性并不意味着视觉接地能力：能够可靠地从文本中标记陈旧条目的模型，在完全相同的网格上其视觉F1分数跨度从0.887低至0.067，且最弱的模型仍持续做出流畅、自信却无视图像的决策。第二，在未经审计的情况下使用陈旧记忆是一种安全隐患：在我们的主要GPT-4o设置中，完全信任原始记忆的智能体的死亡频率是完全没有记忆的同一智能体的两倍以上。第三，审计虽有助于改善但未能弥合差距：透明的读取时过滤器在文本模式下消解了大部分安全代价，然而即使在当前网格规模下引入真值陈旧标签也未带来进一步的显著收益；当视觉审计不可靠时，过滤并未产生一致的增益。综上所述，这些结果将空间记忆的陈旧性界定为一种安全失效模式，并将记忆-观察冲突下可靠的视觉接地与动作选择确立为记忆增强智能体的核心开放挑战。

English

Memory-augmented VLM agents act on persistent spatial knowledge, yet that knowledge silently goes stale as the environment changes. We ask what happens when an agent must reconcile a confident memory claim with a contradicting observation, and whether current models can catch the conflict before it becomes a safety-relevant mistake. Using a dynamic FrozenLake testbed, we pair a staleness-detection task with a downstream navigation task across three closed-source models and three open-weight VLMs under both text and image inputs (1,800 detection runs, and 12,000 text-mode navigation episodes over four LLM navigators at a shared 50-seed scale). Three findings emerge. First, text solvability does not imply visual grounding: models that flag stale entries reliably from text nonetheless span vision F1 from 0.887 down to 0.067 on the identical grids, and the weakest keeps making fluent, confident decisions that ignore the image. Second, consuming stale memory without an audit is a safety liability: in our primary GPT-4o setting, an agent that trusts raw memory dies more than twice as often as the same agent given no memory at all. Third, auditing helps but does not close the gap: a transparent read-time filter removes much of the safety cost in text mode, yet even oracle stale labels bring no further significant gain on the current grid size, and when visual auditing is unreliable, filtering yields no consistent benefit. Together these results frame spatial-memory staleness as a safety failure mode and isolate reliable visual grounding and action selection under memory--observation conflict as the central open challenges for memory-augmented agents.