不确定性位置：大型语言模型中位置偏差的跨语言研究

Position of Uncertainty: A Cross-Linguistic Study of Positional Bias in Large Language Models

May 22, 2025

作者: Menschikov Mikhail, Alexander Kharitonov, Maiia Kotyga, Vadim Porvatov, Anna Zhukovskaya, David Kagramanyan, Egor Shvetsov, Evgeny Burnaev

cs.AI

摘要

大型语言模型展现出位置偏差——即对特定上下文位置信息的系统性忽视——然而其与语言多样性的相互作用仍鲜为人知。我们开展了一项跨语言研究，涵盖五种类型学上迥异的语言（英语、俄语、德语、印地语、越南语），探讨位置偏差如何与模型不确定性、句法结构及提示方式相互影响。主要发现如下：(1) 位置偏差由模型驱动，且存在语言特异性变化——Qwen2.5-7B偏好后置信息，挑战了早期令牌偏差的假设；(2) 明确的定位指导（如，正确信息位于位置X）降低了跨语言的准确性，削弱了提示工程实践的效果；(3) 使上下文与位置偏差对齐会增加熵值，但最低熵值并不能预测准确性。(4) 我们进一步发现，在印地语等自由词序语言中，大型语言模型对主导词序的施加方式有所不同。

English

Large language models exhibit positional bias -- systematic neglect of information at specific context positions -- yet its interplay with linguistic diversity remains poorly understood. We present a cross-linguistic study across five typologically distinct languages (English, Russian, German, Hindi, Vietnamese), examining how positional bias interacts with model uncertainty, syntax, and prompting. Key findings: (1) Positional bias is model-driven, with language-specific variations -- Qwen2.5-7B favors late positions, challenging assumptions of early-token bias; (2) Explicit positional guidance (e.g., correct context is at position X) reduces accuracy across languages, undermining prompt-engineering practices; (3) Aligning context with positional bias increases entropy, yet minimal entropy does not predict accuracy. (4) We further uncover that LLMs differently impose dominant word order in free-word-order languages like Hindi.