大型语言模型的记忆

摘要

记忆已演变为大语言模型（LLM）中一个基础性的架构维度，从计算中隐含的副产品转变为一系列显式、可控的机制。尽管近期研究引入了多样化的策略——涵盖瞬态注意力、循环状态动力学、参数高效适配以及可扩展的查找存储——但这种快速演进导致了高度碎片化的研究格局。本综述提出了一种系统化的、以架构为中心的LLM记忆分类体系。我们的框架沿三个正交维度刻画记忆：表征（隐式与显式）、更新动态（离线与在线）以及持久性（短期与长期）。我们进一步形式化了决定记忆写入、路由、状态转换与整合的粒度化机制。这一统一视角阐明了计算耦合型与独立可寻址记忆之间的概念边界，有效衔接了不同架构范式。此外，我们批判性地分析了混合记忆架构、系统级效率权衡以及多维评估方法。通过将这些分散的进展整合为一个连贯框架，本综述勾勒了以记忆为中心的LLM设计的发展轨迹，并为未来可扩展与自适应语言建模的创新奠定了原理性基础。

English

Memory has evolved into a foundational architectural dimension in large language models (LLMs), shifting from an implicit byproduct of computation to a spectrum of explicit, controllable mechanisms. While recent advances introduce diverse strategies---spanning transient attention, recurrent state dynamics, parameter-efficient adaptations, and scalable lookup storage---this rapid evolution has led to a highly fragmented research landscape. In this survey, we present a systematic, architecture-centric taxonomy of memory in LLMs. Our framework characterizes memory along three orthogonal axes: representation (implicit versus explicit), update dynamics (offline versus online), and persistence (short-term versus long-term). We further formalize the granular mechanisms dictating memory writing, routing, state transitions, and consolidation. This unified perspective elucidates the conceptual boundaries between computation-coupled and independently addressable memory, effectively bridging disparate architectural paradigms. Additionally, we critically analyze hybrid memory architectures, system-level efficiency trade-offs, and multi-dimensional evaluation methodologies. By consolidating these scattered advancements into a cohesive framework, this survey charts the trajectory of memory-centric LLM design and provides a principled foundation for future innovations in scalable and adaptive language modeling.