跨模型記憶遷移：透過目標端讀取器適應

摘要

在大型語言模型中改善知識使用的方法通常可分為兩類範式。非參數檢索能彈性存取外部知識，但會增加檢索延遲、上下文負擔，且與主幹模型的整合僅停留在淺層。參數式調適在推論時效率較高，但會將知識與模型權重糾纏在一起，難以更新、稽核或遷移。Engram 風格的雜湊記憶則屬於中間範式：它將學習到的資訊儲存在外部、可定址的表格中，並透過一個小型學習式讀取器來取用該表格。這引出了一個基本問題：當此類記憶被遷移至不同的主幹模型時，何者更為關鍵？是凍結的記憶本身，還是目標端的讀取器？我們透過跨模型凍結記憶抽取來研究此問題，也就是將在源模型上訓練的記憶凍結，並附加到不同的目標模型上，僅訓練輕量級讀取器。消融實驗顯示，學習到的記憶內容與正確的定址機制皆具重要性，但遷移後的表格唯有透過與目標模型對齊的讀取器，方能發揮效用。在下游問答任務中，雙層四分支讀取器幾乎縮小了同模型重用與跨模型重用之間的差距，在我們受控的評估協議下達到平均分數 38.8。此外，當提供端讀取器能直接相容於目標端介面時，凍結工件無需目標端訓練即可提供實質效用，而選擇性的讀取器調適則能進一步改善表現。這些結果顯示，只要目標端能取得相容的讀取器介面，Engram 便可作為可重用的外部知識工件；當直接重用讀取器不足時，目標端調適可進一步提升對齊程度。

English

Methods for improving knowledge use in large language models typically fall into two regimes. Non-parametric retrieval offers flexible access to external knowledge, but adds retrieval latency, context overhead, and only shallow integration with the backbone. Parametric adaptation is efficient at inference time, but entangles knowledge with model weights and can be hard to update, audit, or transfer. Engram-style hashed memory occupies a middle regime: it stores learned information in an external, addressable table, yet consumes that table through a small learned reader. This raises a basic question: when such a memory is moved across backbones, what matters more, the frozen memory itself or the target-side reader? We study this question through cross-model frozen-memory extraction, in which a memory trained on a source model is frozen and attached to a different target model, with only a lightweight reader trained. Ablations show that learned memory content and correct addressing both matter, but the transferred table becomes useful only through a reader aligned to the target model. In downstream question answering tasks, a dual-layer, four-branch reader nearly closes the gap between same-model and cross-model reuse, achieving an average score of 38.8 under our controlled evaluation protocol. Moreover, when the provider reader is directly compatible with the target interface, the frozen artifact can provide substantial utility without target-side training, while optional reader adaptation yields further improvement. These results suggest that Engram can serve as a reusable external knowledge artifact, provided that the target has access to a compatible reader interface; target-side adaptation can further improve alignment when direct reader reuse is insufficient.