信任與準確性的故事：在RAG系統中的基礎對比指導LLMs

A Tale of Trust and Accuracy: Base vs. Instruct LLMs in RAG Systems

June 21, 2024

作者: Florin Cuconasu, Giovanni Trappolini, Nicola Tonellotto, Fabrizio Silvestri

cs.AI

摘要

檢索增強生成（RAG）代表著人工智慧領域的一項重大進步，結合了檢索階段和生成階段，後者通常由大型語言模型（LLMs）提供動力。目前在RAG中的常見做法包括使用「指導」LLMs，這些模型經過監督訓練進行微調，以增強其遵循指示的能力，並使用最先進的技術與人類偏好保持一致。與普遍觀念相反，我們的研究表明，在我們的實驗設置下，基本模型在RAG任務中平均表現比其指導對應物高出20％。這一發現挑戰了人們對於RAG應用中指導LLMs優越性的普遍假設。進一步的研究揭示了一個更加微妙的情況，質疑了RAG的基本方面，並提出了對該主題進行更廣泛討論的必要性；或者，如弗洛姆所言，“很少有人僅僅透過一瞥統計數據就能理解數字的含義”。

English

Retrieval Augmented Generation (RAG) represents a significant advancement in artificial intelligence combining a retrieval phase with a generative phase, with the latter typically being powered by large language models (LLMs). The current common practices in RAG involve using "instructed" LLMs, which are fine-tuned with supervised training to enhance their ability to follow instructions and are aligned with human preferences using state-of-the-art techniques. Contrary to popular belief, our study demonstrates that base models outperform their instructed counterparts in RAG tasks by 20% on average under our experimental settings. This finding challenges the prevailing assumptions about the superiority of instructed LLMs in RAG applications. Further investigations reveal a more nuanced situation, questioning fundamental aspects of RAG and suggesting the need for broader discussions on the topic; or, as Fromm would have it, "Seldom is a glance at the statistics enough to understand the meaning of the figures".