MemoRAG：通過受記憶啟發的知識發現邁向下一代RAG

MemoRAG: Moving towards Next-Gen RAG Via Memory-Inspired Knowledge Discovery

September 9, 2024

作者: Hongjin Qian, Peitian Zhang, Zheng Liu, Kelong Mao, Zhicheng Dou

cs.AI

摘要

檢索增強生成（RAG）利用檢索工具訪問外部數據庫，從而通過優化上下文提高大型語言模型（LLMs）的生成質量。然而，現有的檢索方法固有地受限，因為它們只能在明確陳述的查詢和格式良好的知識之間進行相關性匹配，無法處理涉及模糊信息需求或非結構化知識的任務。因此，現有的RAG系統主要適用於直接的問答任務。在這項工作中，我們提出了MemoRAG，一種由長期記憶賦能的新型檢索增強生成範式。MemoRAG採用雙系統架構。一方面，它使用輕量但長程LLM來形成數據庫的全局記憶。一旦提出任務，它生成初步答案，提示檢索工具在數據庫中找到有用信息。另一方面，它利用昂貴但表達豐富的LLM，基於檢索到的信息生成最終答案。在這個通用框架基礎上，我們通過增強其提示機制和記憶容量進一步優化MemoRAG的性能。在我們的實驗中，MemoRAG在各種評估任務中取得優異表現，包括傳統RAG失敗的複雜任務和RAG常應用的直接任務。

English

Retrieval-Augmented Generation (RAG) leverages retrieval tools to access external databases, thereby enhancing the generation quality of large language models (LLMs) through optimized context. However, the existing retrieval methods are constrained inherently, as they can only perform relevance matching between explicitly stated queries and well-formed knowledge, but unable to handle tasks involving ambiguous information needs or unstructured knowledge. Consequently, existing RAG systems are primarily effective for straightforward question-answering tasks. In this work, we propose MemoRAG, a novel retrieval-augmented generation paradigm empowered by long-term memory. MemoRAG adopts a dual-system architecture. On the one hand, it employs a light but long-range LLM to form the global memory of database. Once a task is presented, it generates draft answers, cluing the retrieval tools to locate useful information within the database. On the other hand, it leverages an expensive but expressive LLM, which generates the ultimate answer based on the retrieved information. Building on this general framework, we further optimize MemoRAG's performance by enhancing its cluing mechanism and memorization capacity. In our experiment, MemoRAG achieves superior performance across a variety of evaluation tasks, including both complex ones where conventional RAG fails and straightforward ones where RAG is commonly applied.