一个层解释所有：理解大型语言模型中的大规模激活

摘要

我们研究了大语言模型（LLMs）中大规模激活的起源，并识别出一个被称为大规模涌现层（ME Layer）的特定层——该层在不同模型家族中均稳定存在，其中大规模激活首先在此出现，随后通过残差连接向更深层传播。研究表明，在ME层内，RMSNorm与前馈网络（FFN）参数共同促成了大规模激活的产生。一旦形成，大规模激活的令牌表示在跨层传递时基本保持不变，从而降低了传递给注意力模块的隐藏表示多样性。受此局限性的启发，我们提出了一种简单有效的方法来降低大规模激活令牌的刚性。该方法在无需训练和微调两种场景下，均能持续提升大语言模型在指令遵循和数学推理等多项任务上的性能。此外，我们证明该方法通过选择性削弱注意力沉点的权重来缓解其影响，从隐藏状态层面揭示了注意力沉点的起源，并为原则性缓解策略提供了新见解。

English

We investigate the origins of massive activations in large language models (LLMs) and identify a specific layer named the Massive Emergence Layer (ME Layer), that is consistently observed across model families, where massive activations first emerge and subsequently propagate to deeper layers through residual connections. We show that, within the ME Layer both the RMSNorm and the FFN parameters jointly contribute to the emergence of massive activations. Once formed, the massive activation token representation remains largely invariant across layers, reducing the diversity of hidden representations passed to the attention module. Motivated by this limitation, we propose a simple and effective method to reduce the rigidity of the massive activation token. Our approach consistently improves LLM performance across multiple tasks, including instruction following and math reasoning, in both training free and fine tuning settings. Moreover, we show that our method mitigates attention sinks by selectively weakening their influence, elucidating their origin at the hidden state level and shedding new light on principled mitigation strategies.

一个层解释所有：理解大型语言模型中的大规模激活

A Single Layer to Explain Them All:Understanding Massive Activations in Large Language Models

摘要

Support