注意力溢出：長文本情境下的語言模型輸入模糊 缺失項目建議

Attention Overflow: Language Model Input Blur during Long-Context Missing Items Recommendation

July 18, 2024

作者: Damien Sileo

cs.AI

摘要

大型語言模型（LLMs）可以從提示中列出的項目中提供建議缺失的元素，這可用於完成列表或基於用戶歷史的推薦。然而，當它們面對太多項目時，性能會下降，因為它們開始建議已包含在輸入列表中的項目。這在2024年中期旗艦LLMs的情況下大約發生在100個項目左右。我們在合成問題（例如，在打亂的整數範圍中查找缺失的數字）和現實電影推薦情境中評估這一現象。我們將這個問題稱為注意力溢出，因為防止重複需要同時關注所有項目。儘管迭代循環可以緩解這個問題，但它們的成本隨著重複率的增加而增加，影響語言模型從冗長輸入中獲得新穎性的能力。

English

Large language models (LLMs) can suggest missing elements from items listed in a prompt, which can be used for list completion or recommendations based on users' history. However, their performance degrades when presented with too many items, as they start to suggest items already included in the input list. This occurs at around 100 items for mid-2024 flagship LLMs. We evaluate this phenomenon on both synthetic problems (e.g., finding missing numbers in a given range of shuffled integers) and realistic movie recommendation scenarios. We refer to this issue as attention overflow, as preventing repetition requires attending to all items simultaneously. Although iterative loops can mitigate this problem, their costs increase with the repetition rate, affecting the language models' ability to derive novelty from lengthy inputs.