注意力溢出：长上下文期间语言模型输入模糊 缺失项目推荐

Attention Overflow: Language Model Input Blur during Long-Context Missing Items Recommendation

July 18, 2024

作者: Damien Sileo

cs.AI

摘要

大型语言模型（LLMs）可以从提示中列出的项目中提供建议缺失的元素，这可以用于完成列表或基于用户历史记录进行推荐。然而，当呈现太多项目时，它们的性能会下降，因为它们开始建议已包含在输入列表中的项目。这种情况在2024年中期的旗舰LLMs中大约100个项目时发生。我们在合成问题（例如，在打乱的整数范围中查找缺失数字）和现实电影推荐场景中评估了这种现象。我们将这个问题称为注意力溢出，因为防止重复需要同时关注所有项目。尽管迭代循环可以减轻这个问题，但它们的成本随着重复率的增加而增加，影响语言模型从冗长输入中提取新颖性的能力。

English

Large language models (LLMs) can suggest missing elements from items listed in a prompt, which can be used for list completion or recommendations based on users' history. However, their performance degrades when presented with too many items, as they start to suggest items already included in the input list. This occurs at around 100 items for mid-2024 flagship LLMs. We evaluate this phenomenon on both synthetic problems (e.g., finding missing numbers in a given range of shuffled integers) and realistic movie recommendation scenarios. We refer to this issue as attention overflow, as preventing repetition requires attending to all items simultaneously. Although iterative loops can mitigate this problem, their costs increase with the repetition rate, affecting the language models' ability to derive novelty from lengthy inputs.