不確定性位置：大型語言模型中位置偏差的跨語言研究

Position of Uncertainty: A Cross-Linguistic Study of Positional Bias in Large Language Models

May 22, 2025

作者: Menschikov Mikhail, Alexander Kharitonov, Maiia Kotyga, Vadim Porvatov, Anna Zhukovskaya, David Kagramanyan, Egor Shvetsov, Evgeny Burnaev

cs.AI

摘要

大型語言模型展現出位置偏見——即系統性地忽略特定上下文位置的信息——然而其與語言多樣性之間的相互作用仍鮮為人知。我們進行了一項跨語言研究，涵蓋五種類型學上截然不同的語言（英語、俄語、德語、印地語、越南語），探討位置偏見如何與模型不確定性、句法及提示方式相互作用。主要發現如下：(1) 位置偏見由模型驅動，並呈現語言特異性變化——Qwen2.5-7B偏好後期位置，挑戰了早期詞彙偏見的假設；(2) 明確的位置指導（例如，正確上下文位於位置X）降低了跨語言的準確性，削弱了提示工程實踐；(3) 將上下文與位置偏見對齊會增加熵值，但最小熵值並不能預測準確性。(4) 我們進一步發現，在印地語等自由詞序語言中，LLMs以不同方式強加主導詞序。

English

Large language models exhibit positional bias -- systematic neglect of information at specific context positions -- yet its interplay with linguistic diversity remains poorly understood. We present a cross-linguistic study across five typologically distinct languages (English, Russian, German, Hindi, Vietnamese), examining how positional bias interacts with model uncertainty, syntax, and prompting. Key findings: (1) Positional bias is model-driven, with language-specific variations -- Qwen2.5-7B favors late positions, challenging assumptions of early-token bias; (2) Explicit positional guidance (e.g., correct context is at position X) reduces accuracy across languages, undermining prompt-engineering practices; (3) Aligning context with positional bias increases entropy, yet minimal entropy does not predict accuracy. (4) We further uncover that LLMs differently impose dominant word order in free-word-order languages like Hindi.