拉里瑪：具有情節記憶控制的大型語言模型

Larimar: Large Language Models with Episodic Memory Control

March 18, 2024

作者: Payel Das, Subhajit Chaudhury, Elliot Nelson, Igor Melnyk, Sarath Swaminathan, Sihui Dai, Aurélie Lozano, Georgios Kollias, Vijil Chenthamarakshan, Jiří, Navrátil, Soham Dan, Pin-Yu Chen

cs.AI

摘要

在當今，有效且準確地更新存儲在大型語言模型（LLMs）中的知識是最迫切的研究挑戰之一。本文介紹了Larimar - 一種新穎的、受大腦啟發的架構，用於通過分佈式情景記憶來增強LLMs。Larimar的記憶允許動態、一次性地更新知識，無需進行計算昂貴的重新訓練或微調。在多個事實編輯基準測試中的實驗結果表明，Larimar實現了與大多數競爭基準相當的準確性，即使在具有挑戰性的順序編輯設置中，也表現出色，同時在速度方面優勢明顯 - 根據基礎LLM的不同，加速效果為4-10倍 - 並且由於所提出的架構簡單、與LLM無關，因此具有通用性。我們進一步提供了選擇性事實遺忘和輸入上下文長度泛化的機制，並展示了它們的有效性。

English

Efficient and accurate updating of knowledge stored in Large Language Models (LLMs) is one of the most pressing research challenges today. This paper presents Larimar - a novel, brain-inspired architecture for enhancing LLMs with a distributed episodic memory. Larimar's memory allows for dynamic, one-shot updates of knowledge without the need for computationally expensive re-training or fine-tuning. Experimental results on multiple fact editing benchmarks demonstrate that Larimar attains accuracy comparable to most competitive baselines, even in the challenging sequential editing setup, but also excels in speed - yielding speed-ups of 4-10x depending on the base LLM - as well as flexibility due to the proposed architecture being simple, LLM-agnostic, and hence general. We further provide mechanisms for selective fact forgetting and input context length generalization with Larimar and show their effectiveness.