RAG系統是否存在位置偏差？

Do RAG Systems Suffer From Positional Bias?

May 21, 2025

作者: Florin Cuconasu, Simone Filice, Guy Horowitz, Yoelle Maarek, Fabrizio Silvestri

cs.AI

摘要

檢索增強生成技術通過將從外部語料庫檢索到的段落添加到大型語言模型（LLM）的提示中，從而提升其準確性。本文探討了位置偏差——即LLM根據信息在提示中的位置給予不同權重的傾向——如何不僅影響LLM利用相關段落的能力，還影響其對干擾段落的敏感性。通過在三個基準上的廣泛實驗，我們展示了最先進的檢索管道在嘗試檢索相關段落的同時，系統性地將高度干擾的段落帶到排名前列，超過60%的查詢在其前10個檢索段落中至少包含一個高度干擾的段落。因此，在受控環境中常被相關研究報告為非常顯著的LLM位置偏差，在實際場景中的影響實際上微乎其微，因為相關和干擾段落都會相應地受到懲罰。事實上，我們的研究發現，試圖根據LLM位置偏好重新排列段落的複雜策略，其表現並不比隨機打亂更好。

English

Retrieval Augmented Generation enhances LLM accuracy by adding passages retrieved from an external corpus to the LLM prompt. This paper investigates how positional bias - the tendency of LLMs to weight information differently based on its position in the prompt - affects not only the LLM's capability to capitalize on relevant passages, but also its susceptibility to distracting passages. Through extensive experiments on three benchmarks, we show how state-of-the-art retrieval pipelines, while attempting to retrieve relevant passages, systematically bring highly distracting ones to the top ranks, with over 60% of queries containing at least one highly distracting passage among the top-10 retrieved passages. As a result, the impact of the LLM positional bias, which in controlled settings is often reported as very prominent by related works, is actually marginal in real scenarios since both relevant and distracting passages are, in turn, penalized. Indeed, our findings reveal that sophisticated strategies that attempt to rearrange the passages based on LLM positional preferences do not perform better than random shuffling.