AtlasVA：面向無教師VLM代理的自演化視覺技能記憶

摘要

视觉语言模型（VLM）智能体日益依赖记忆增强强化学习来在长时程任务中复用经验，然而现有框架大多以文本形式存储记忆，并依赖专有教师模型来总结或精炼这些记忆。这种设计与空间决策任务并不匹配：几何先验信息被压缩为有损的语言描述，稀疏的交互过程往往通过延迟的文本反馈而非密集的视觉化信号进行监督。我们认为，VLM智能体的可复用经验应当保持视觉化基础。基于这一洞察，我们提出AtlasVA——一种无需教师模型的视觉技能记忆框架，它将记忆组织为三个互补层次：空间热力图、视觉示例和符号化文本技能。AtlasVA进一步通过轨迹统计和轻量级网格启发式方法直接演化出危险图谱与亲和力图谱，并将这些自演化图谱用作基于势的塑形奖励，以支持强化学习。这一框架统一了感知、记忆与优化，无需外部大语言模型监督。在Sokoban、FrozenLake、3D具身导航和3D机器人操作基准上的实验表明，AtlasVA始终优于以文本为中心的记忆基线和有竞争力的VLM智能体，尤其在空间密集型任务上表现突出。主页：https://wangpan-ustc.github.io/AtlasvaWeb

English

Vision-language model (VLM) agents increasingly rely on memory-augmented reinforcement learning to reuse experience across long-horizon tasks, yet most existing frameworks store memory as text and depend on proprietary teacher models to summarize or refine it. This design is poorly matched to spatial decision making: geometric priors are compressed into lossy language, and sparse interaction is often supervised through delayed textual feedback rather than dense visually grounded signals. We argue that reusable experience for VLM agents should remain visually grounded. Based on this insight, we propose AtlasVA, a teacher-free visual skill memory framework that organizes memory into three complementary layers: spatial heatmaps, visual exemplars, and symbolic text skills. AtlasVA further evolves danger and affinity atlases directly from trajectory statistics and lightweight grid heuristics, and reuses these self-evolving atlases as potential-based shaping rewards for reinforcement learning. This unifies perception, memory, and optimization without external LLM supervision. Experiments on Sokoban, FrozenLake, 3D embodied navigation, and 3D robotic manipulation benchmarks show that AtlasVA consistently outperforms text-centric memory baselines and competitive VLM agents, with especially strong gains on spatially intensive tasks. Homepage: https://wangpan-ustc.github.io/AtlasvaWeb