RAG系统是否存在位置偏差？

Do RAG Systems Suffer From Positional Bias?

May 21, 2025

作者: Florin Cuconasu, Simone Filice, Guy Horowitz, Yoelle Maarek, Fabrizio Silvestri

cs.AI

摘要

检索增强生成技术通过将外部语料库中检索到的段落加入大语言模型（LLM）的提示中，提升了其准确性。本文探讨了位置偏差——即LLM根据信息在提示中的位置赋予不同权重的倾向——如何不仅影响LLM利用相关段落的能力，还影响其对干扰段落的敏感性。通过在三个基准数据集上的大量实验，我们发现，尽管最先进的检索流程旨在获取相关段落，却系统性地将高度干扰的段落推至前列，超过60%的查询在其前10个检索段落中至少包含一个高度干扰的段落。因此，在受控环境中常被相关研究报道为非常显著的位置偏差效应，在实际场景中其实影响甚微，因为相关段落和干扰段落同样受到了抑制。实际上，我们的研究结果表明，试图根据LLM的位置偏好重新排列段落的复杂策略，其表现并不优于随机打乱顺序。

English

Retrieval Augmented Generation enhances LLM accuracy by adding passages retrieved from an external corpus to the LLM prompt. This paper investigates how positional bias - the tendency of LLMs to weight information differently based on its position in the prompt - affects not only the LLM's capability to capitalize on relevant passages, but also its susceptibility to distracting passages. Through extensive experiments on three benchmarks, we show how state-of-the-art retrieval pipelines, while attempting to retrieve relevant passages, systematically bring highly distracting ones to the top ranks, with over 60% of queries containing at least one highly distracting passage among the top-10 retrieved passages. As a result, the impact of the LLM positional bias, which in controlled settings is often reported as very prominent by related works, is actually marginal in real scenarios since both relevant and distracting passages are, in turn, penalized. Indeed, our findings reveal that sophisticated strategies that attempt to rearrange the passages based on LLM positional preferences do not perform better than random shuffling.