MemTrace：大型語言模型記憶系統中的錯誤追蹤與歸因

摘要

記憶對於讓大型語言模型支援長程推理至關重要，然而現有的記憶系統仍不可靠且難以除錯。追蹤記憶的動態演化對於理解資訊如何隨時間合成、傳播或受損至關重要。在本研究中，我們探討大型語言模型記憶系統中錯誤追蹤與歸因的新問題。我們提出一個新穎框架，將記憶管線轉換為可執行的記憶演化圖，從而實現對操作資訊流的細粒度追蹤。接著我們建構了 MemTraceBench，這是一個從代表性記憶系統（如 Long-Context、RAG、Mem0 和 EverMemOS）收集的基準測試，用以系統性研究記憶失效模式。我們進一步引入一種自動歸因方法，透過迭代追蹤操作子圖，來精確定位任何失敗案例的根本原因。我們的分析揭示，記憶失效是系統性的，源於操作層面的問題，如資訊遺失與檢索錯位。關鍵的是，我們利用這些細粒度歸因訊號來引導下游提示優化，建立了一個自動修正錯誤並將最終任務效能提升高達 7.62% 的閉環系統。程式碼將於 https://github.com/zjunlp/MemTrace 釋出。

English

Memory is essential for enabling large language models to support long-horizon reasoning, yet existing memory systems remain unreliable and difficult to debug. Tracing memory's dynamic evolution is crucial to understand how information is synthesized, propagated, or corrupted over time. In this work, we study the new problem of error tracing and attribution in LLM memory systems. We propose a novel framework that transforms memory pipelines into executable memory evolution graphs, enabling fine-grained tracing of operational information flow. We then construct MemTraceBench, a benchmark collected from representative memory systems such as Long-Context, RAG, Mem0, and EverMemOS, to systematically study memory failure modes. We further introduce an automatic attribution method that iteratively traces operation subgraphs to pinpoint the root cause of any failed case. Our analysis reveals that memory failures are systematic, stemming from operation-level issues like information loss and retrieval misalignment. Crucially, we leverage these fine-grained attribution signals to guide downstream prompt optimization, establishing a closed-loop system that automatically corrects faults and boosts end-task performance by up to 7.62%. Code will be released at https://github.com/zjunlp/MemTrace.