跨模型记忆迁移：通过目标端阅读器适配

摘要

改进大语言模型中知识使用的方法通常分为两类。非参数检索能够灵活访问外部知识，但会引入检索延迟、上下文开销，并且与主干模型的集成仅停留在浅层。参数化适配在推理时高效，但将知识与模型权重耦合在一起，难以更新、审计或迁移。Engram式哈希记忆则介于两者之间：它将学习到的信息存储在外部的可寻址表中，但通过一个小型的学习型读取器来消费该表。这引出了一个基本问题：当这样的记忆跨主干模型迁移时，冻结的记忆本身与目标侧的读取器，哪一个更重要？我们通过跨模型冻结记忆提取来研究这一问题，即将在源模型上训练好的记忆冻结，并附着到不同的目标模型上，仅训练一个轻量级读取器。消融实验表明，学习到的记忆内容和正确的寻址都很重要，但迁移过来的表只有在读取器与目标模型对齐时才能发挥作用。在下游问答任务中，一个双层、四分支读取器几乎弥合了同模型复用与跨模型复用之间的差距，在我们受控的评估协议下达到了38.8的平均得分。此外，当提供方读取器与目标接口直接兼容时，冻结产物无需目标侧训练即可提供可观效用，而可选的读取器适配则能带来进一步提升。这些结果表明，只要目标方能够访问兼容的读取器接口，Engram即可作为可重用的外部知识产物；当直接复用读取器不足时，目标侧适配可进一步改善对齐效果。

English

Methods for improving knowledge use in large language models typically fall into two regimes. Non-parametric retrieval offers flexible access to external knowledge, but adds retrieval latency, context overhead, and only shallow integration with the backbone. Parametric adaptation is efficient at inference time, but entangles knowledge with model weights and can be hard to update, audit, or transfer. Engram-style hashed memory occupies a middle regime: it stores learned information in an external, addressable table, yet consumes that table through a small learned reader. This raises a basic question: when such a memory is moved across backbones, what matters more, the frozen memory itself or the target-side reader? We study this question through cross-model frozen-memory extraction, in which a memory trained on a source model is frozen and attached to a different target model, with only a lightweight reader trained. Ablations show that learned memory content and correct addressing both matter, but the transferred table becomes useful only through a reader aligned to the target model. In downstream question answering tasks, a dual-layer, four-branch reader nearly closes the gap between same-model and cross-model reuse, achieving an average score of 38.8 under our controlled evaluation protocol. Moreover, when the provider reader is directly compatible with the target interface, the frozen artifact can provide substantial utility without target-side training, while optional reader adaptation yields further improvement. These results suggest that Engram can serve as a reusable external knowledge artifact, provided that the target has access to a compatible reader interface; target-side adaptation can further improve alignment when direct reader reuse is insufficient.