OpenDecoder：开放大型语言模型解码以在RAG中融入文档质量评估

摘要

大型语言模型（LLM）的发展在一系列下游任务中取得了卓越性能，其中包括基于LLM的检索增强生成（RAG）。生成内容的质量高度依赖于检索信息的有用性，以及LLM内部信息处理机制在答案生成中整合这些信息的能力。通常假设检索到的信息与问题相关，但实际检索信息的关联度和有用性会因问题及文档集合的不同而存在差异。因此在答案生成过程中考虑检索信息的相关性至关重要。本文提出OpenDecoder新方法，通过显式评估检索信息作为生成质量指标特征，旨在构建对不同程度噪声上下文具有更强鲁棒性的RAG模型。我们综合考虑三类显式评估信息：相关性评分、排序评分和QPP（查询性能预测）评分。在五个基准数据集上的实验结果表明，OpenDecoder以超越多种基线方法的性能验证了其有效性与更优的鲁棒性。值得注意的是，该范式具有高度灵活性，既可适配LLM针对不同目标的后训练任务，也能兼容各类外部指标。

English

The development of large language models (LLMs) has achieved superior performance in a range of downstream tasks, including LLM-based retrieval-augmented generation (RAG). The quality of generated content heavily relies on the usefulness of the retrieved information and the capacity of LLMs' internal information processing mechanism to incorporate it in answer generation. It is generally assumed that the retrieved information is relevant to the question. However, the retrieved information may have a variable degree of relevance and usefulness, depending on the question and the document collection. It is important to take into account the relevance of the retrieved information in answer generation. In this paper, we propose OpenDecoder, a new approach that leverages explicit evaluation of the retrieved information as quality indicator features for generation. We aim to build a RAG model that is more robust to varying levels of noisy context. Three types of explicit evaluation information are considered: relevance score, ranking score, and QPP (query performance prediction) score. The experimental results on five benchmark datasets demonstrate the effectiveness and better robustness of OpenDecoder by outperforming various baseline methods. Importantly, this paradigm is flexible to be integrated with the post-training of LLMs for any purposes and incorporated with any type of external indicators.

OpenDecoder：开放大型语言模型解码以在RAG中融入文档质量评估

OpenDecoder: Open Large Language Model Decoding to Incorporate Document Quality in RAG

摘要

Support