OpenDecoder：開放式大型語言模型解碼技術——將文件品質納入檢索增強生成的新方法

摘要

大型語言模型（LLM）的發展已在一系列下游任務中取得卓越性能，包括基於LLM的檢索增強生成（RAG）。生成內容的質量高度依賴於檢索信息的實用性，以及LLM內部信息處理機制在答案生成中整合這些信息的能力。通常假設檢索到的信息與問題相關，但實際上檢索信息的相關性和實用性會因問題和文檔集合的不同而存在差異。因此，在答案生成中考慮檢索信息的相關性至關重要。本文提出OpenDecoder——一種新方法，利用對檢索信息的顯式評估作為生成過程中的質量指標特徵。我們的目標是構建一個對不同程度噪聲上下文更具魯棒性的RAG模型。該方法考慮了三類顯式評估信息：相關性評分、排序評分和QPP（查詢性能預測）評分。在五個基準數據集上的實驗結果表明，OpenDecoder通過超越多種基準方法，展現出卓越的有效性與更優的魯棒性。重要的是，此範式具備靈活性，可與任何目的的LLM後訓練相結合，並能整合任意類型的外部指標。

English

The development of large language models (LLMs) has achieved superior performance in a range of downstream tasks, including LLM-based retrieval-augmented generation (RAG). The quality of generated content heavily relies on the usefulness of the retrieved information and the capacity of LLMs' internal information processing mechanism to incorporate it in answer generation. It is generally assumed that the retrieved information is relevant to the question. However, the retrieved information may have a variable degree of relevance and usefulness, depending on the question and the document collection. It is important to take into account the relevance of the retrieved information in answer generation. In this paper, we propose OpenDecoder, a new approach that leverages explicit evaluation of the retrieved information as quality indicator features for generation. We aim to build a RAG model that is more robust to varying levels of noisy context. Three types of explicit evaluation information are considered: relevance score, ranking score, and QPP (query performance prediction) score. The experimental results on five benchmark datasets demonstrate the effectiveness and better robustness of OpenDecoder by outperforming various baseline methods. Importantly, this paradigm is flexible to be integrated with the post-training of LLMs for any purposes and incorporated with any type of external indicators.

OpenDecoder：開放式大型語言模型解碼技術——將文件品質納入檢索增強生成的新方法

OpenDecoder: Open Large Language Model Decoding to Incorporate Document Quality in RAG

摘要

Support