大型语言模型中的神经元：死亡、N-gram、位置化

摘要

我们以一种轻量级的方式分析了一类大型语言模型，这种分析可以在单个GPU上完成。具体来说，我们关注参数范围从125m到66b的OPT模型系列，仅依赖于前馈神经网络（FFN）神经元是否被激活。首先，我们发现网络的前部稀疏且代表许多离散特征。在这里，许多神经元（在66b模型的某些层中超过70%）是“死”的，即它们在大量多样化数据上从不被激活。与此同时，许多活跃的神经元被保留用于离散特征，并充当标记和n-gram检测器。有趣的是，它们对应的FFN更新不仅促进下一个标记候选项，这是可以预期的，而且明确专注于消除有关触发这些标记的信息，即当前输入。据我们所知，这是专门用于从残余流中删除（而不是添加）信息的机制的首个示例。随着规模的扩大，模型在某种意义上变得更加稀疏，即有更多的死神经元和标记检测器。最后，一些神经元是位置相关的：它们是否被激活在很大程度上（或完全）取决于位置，而不那么（或根本不）取决于文本数据。我们发现较小的模型具有一组神经元充当位置范围指示器，而较大的模型以一种不那么明确的方式运作。

English

We analyze a family of large language models in such a lightweight manner that can be done on a single GPU. Specifically, we focus on the OPT family of models ranging from 125m to 66b parameters and rely only on whether an FFN neuron is activated or not. First, we find that the early part of the network is sparse and represents many discrete features. Here, many neurons (more than 70% in some layers of the 66b model) are "dead", i.e. they never activate on a large collection of diverse data. At the same time, many of the alive neurons are reserved for discrete features and act as token and n-gram detectors. Interestingly, their corresponding FFN updates not only promote next token candidates as could be expected, but also explicitly focus on removing the information about triggering them tokens, i.e., current input. To the best of our knowledge, this is the first example of mechanisms specialized at removing (rather than adding) information from the residual stream. With scale, models become more sparse in a sense that they have more dead neurons and token detectors. Finally, some neurons are positional: them being activated or not depends largely (or solely) on position and less so (or not at all) on textual data. We find that smaller models have sets of neurons acting as position range indicators while larger models operate in a less explicit manner.

大型语言模型中的神经元：死亡、N-gram、位置化

Neurons in Large Language Models: Dead, N-gram, Positional

摘要

Support