信任与准确性的故事：RAG系统中的基础LLM与指导LLM

A Tale of Trust and Accuracy: Base vs. Instruct LLMs in RAG Systems

June 21, 2024

作者: Florin Cuconasu, Giovanni Trappolini, Nicola Tonellotto, Fabrizio Silvestri

cs.AI

摘要

检索增强生成（RAG）代表了人工智能领域的重大进展，将检索阶段与生成阶段结合在一起，后者通常由大型语言模型（LLMs）驱动。目前RAG的常见做法涉及使用“指导型”LLMs，这些模型经过监督训练微调，以增强它们遵循指令的能力，并利用最先进的技术与人类偏好保持一致。与普遍看法相反，我们的研究表明，在我们的实验设置下，基础模型在RAG任务中的表现平均比经过指导的模型高出20%。这一发现挑战了人们对于RAG应用中指导型LLMs优越性的普遍假设。进一步的调查揭示了一个更加微妙的情况，质疑了RAG的基本方面，并建议对该主题进行更广泛的讨论；或者，正如弗洛姆所说，“很少有一个对统计数据的一瞥足以理解数字的含义”。

English

Retrieval Augmented Generation (RAG) represents a significant advancement in artificial intelligence combining a retrieval phase with a generative phase, with the latter typically being powered by large language models (LLMs). The current common practices in RAG involve using "instructed" LLMs, which are fine-tuned with supervised training to enhance their ability to follow instructions and are aligned with human preferences using state-of-the-art techniques. Contrary to popular belief, our study demonstrates that base models outperform their instructed counterparts in RAG tasks by 20% on average under our experimental settings. This finding challenges the prevailing assumptions about the superiority of instructed LLMs in RAG applications. Further investigations reveal a more nuanced situation, questioning fundamental aspects of RAG and suggesting the need for broader discussions on the topic; or, as Fromm would have it, "Seldom is a glance at the statistics enough to understand the meaning of the figures".