相似性并非唯一所需：赋予检索增强生成具有多层思维

Similarity is Not All You Need: Endowing Retrieval Augmented Generation with Multi Layered Thoughts

May 30, 2024

作者: Chunjing Gan, Dan Yang, Binbin Hu, Hanxiao Zhang, Siyuan Li, Ziqi Liu, Yue Shen, Lin Ju, Zhiqiang Zhang, Jinjie Gu, Lei Liang, Jun Zhou

cs.AI

摘要

近年来，大型语言模型（LLMs）在各个领域取得了显著成就。然而，LLMs的知识更新及成本问题，以及幻觉问题限制了它们在知识密集型任务中的应用，而检索增强生成（RAG）则可以提供帮助。然而，现有的检索增强模型通常使用相似度作为查询和文档之间的桥梁，并遵循检索然后阅读的过程。在这项工作中，我们认为相似度并非总是灵丹妙药，完全依赖相似度有时会降低检索增强生成的性能。因此，我们提出了MetRag，一种多层思维增强的检索增强生成框架。首先，除了现有的相似度导向思维外，我们采用了一个小规模的效用模型，从LLM中获得效用导向思维的监督，并通过全面结合相似度和效用导向思维提出了更智能的模型。此外，考虑到检索到的文档集往往庞大，并且单独使用它们很难捕捉它们之间的共性和特征，我们提出将LLM作为任务自适应摘要生成器，赋予检索增强生成以紧凑导向思维。最后，通过前述阶段的多层思维，需要调用LLM进行知识增强生成。对知识密集型任务的大量实验表明了MetRag的优越性。

English

In recent years, large language models (LLMs) have made remarkable achievements in various domains. However, the untimeliness and cost of knowledge updates coupled with hallucination issues of LLMs have curtailed their applications in knowledge intensive tasks, where retrieval augmented generation (RAG) can be of help. Nevertheless, existing retrieval augmented models typically use similarity as a bridge between queries and documents and follow a retrieve then read procedure. In this work, we argue that similarity is not always the panacea and totally relying on similarity would sometimes degrade the performance of retrieval augmented generation. To this end, we propose MetRag, a Multi layEred Thoughts enhanced Retrieval Augmented Generation framework. To begin with, beyond existing similarity oriented thought, we embrace a small scale utility model that draws supervision from an LLM for utility oriented thought and further come up with a smarter model by comprehensively combining the similarity and utility oriented thoughts. Furthermore, given the fact that the retrieved document set tends to be huge and using them in isolation makes it difficult to capture the commonalities and characteristics among them, we propose to make an LLM as a task adaptive summarizer to endow retrieval augmented generation with compactness-oriented thought. Finally, with multi layered thoughts from the precedent stages, an LLM is called for knowledge augmented generation. Extensive experiments on knowledge-intensive tasks have demonstrated the superiority of MetRag.