一層解釋全局：理解大型語言模型中的巨量激活

摘要

我們研究了大型語言模型（LLM）中巨量激活的起源，並識別出一個特定層，稱為大量湧現層（ME 層），該層在不同模型家族中一致被觀察到，巨量激活首先在此處湧現，隨後通過殘差連接傳播到更深層。我們表明，在 ME 層內，RMSNorm 和 FFN 參數共同促成了巨量激活的湧現。一旦形成，巨量激活的詞元表示在各層之間保持高度不變，降低了傳遞給注意力模組的隱藏表示的多樣性。基於此限制，我們提出一種簡單有效的方法來減少巨量激活詞元的剛性。我們的方法在多項任務（包括指令遵循和數學推理）中，在免訓練和微調設置下均持續提升了 LLM 的效能。此外，我們展示了我們的方法通過選擇性削弱注意力匯集的影響來緩解注意力匯集問題，闡明了其在隱藏狀態層面的起源，並為原則性緩解策略提供了新見解。

English

We investigate the origins of massive activations in large language models (LLMs) and identify a specific layer named the Massive Emergence Layer (ME Layer), that is consistently observed across model families, where massive activations first emerge and subsequently propagate to deeper layers through residual connections. We show that, within the ME Layer both the RMSNorm and the FFN parameters jointly contribute to the emergence of massive activations. Once formed, the massive activation token representation remains largely invariant across layers, reducing the diversity of hidden representations passed to the attention module. Motivated by this limitation, we propose a simple and effective method to reduce the rigidity of the massive activation token. Our approach consistently improves LLM performance across multiple tasks, including instruction following and math reasoning, in both training free and fine tuning settings. Moreover, we show that our method mitigates attention sinks by selectively weakening their influence, elucidating their origin at the hidden state level and shedding new light on principled mitigation strategies.

一層解釋全局：理解大型語言模型中的巨量激活

A Single Layer to Explain Them All:Understanding Massive Activations in Large Language Models

摘要

Support