第二系统注意力（也许你也需要）

System 2 Attention (is something you might need too)

November 20, 2023

作者: Jason Weston, Sainbayar Sukhbaatar

cs.AI

摘要

基于Transformer的大型语言模型（LLMs）中的软注意力容易将上下文中的无关信息纳入其潜在表示中，从而对下一个标记的生成产生不利影响。为了帮助纠正这些问题，我们引入了System 2 Attention（S2A），它利用LLMs在自然语言推理和遵循指令方面的能力，决定要关注什么。S2A重新生成输入上下文，只包括相关部分，然后关注重新生成的上下文以引出最终响应。在实验中，S2A在包含观点或无关信息的三个任务中表现优于基于标准注意力的LLMs，包括问答、数学文字问题和长篇生成，其中S2A增加了事实性和客观性，减少了阿谀奉承。

English

Soft attention in Transformer-based Large Language Models (LLMs) is susceptible to incorporating irrelevant information from the context into its latent representations, which adversely affects next token generations. To help rectify these issues, we introduce System 2 Attention (S2A), which leverages the ability of LLMs to reason in natural language and follow instructions in order to decide what to attend to. S2A regenerates the input context to only include the relevant portions, before attending to the regenerated context to elicit the final response. In experiments, S2A outperforms standard attention-based LLMs on three tasks containing opinion or irrelevant information, QA, math word problems and longform generation, where S2A increases factuality and objectivity, and decreases sycophancy.