大型語言模型中的神經元：死亡、N-gram、位置化

摘要

我們以一種輕量級的方式分析了一個大型語言模型家族，可以在單個GPU上完成。具體來說，我們專注於OPT家族的模型，其參數範圍從1.25億到660億，僅依賴於FFN神經元是否被激活。首先，我們發現網絡的早期部分是稀疏的，並代表許多離散特徵。在這裡，許多神經元（在660億模型的某些層中超過70％）是“死”的，即它們在大量多樣化數據集上從不被激活。與此同時，許多活躍的神經元被保留用於離散特徵，並充當標記和n-gram檢測器。有趣的是，它們對應的FFN更新不僅促進下一個標記候選項，這是可以預期的，而且還明確專注於刪除有關觸發它們標記的信息，即當前輸入。據我們所知，這是專門用於從剩餘流中刪除（而不是添加）信息的機制的第一個例子。隨著規模的擴大，模型在某種意義上變得更加稀疏，即具有更多死神經元和標記檢測器。最後，一些神經元是位置性的：它們是否被激活在很大程度上（或僅僅）取決於位置，而不那麼（或根本不）取決於文本數據。我們發現較小的模型具有一組神經元作為位置範圍指示器，而較大的模型以一種不那麼明確的方式運作。

English

We analyze a family of large language models in such a lightweight manner that can be done on a single GPU. Specifically, we focus on the OPT family of models ranging from 125m to 66b parameters and rely only on whether an FFN neuron is activated or not. First, we find that the early part of the network is sparse and represents many discrete features. Here, many neurons (more than 70% in some layers of the 66b model) are "dead", i.e. they never activate on a large collection of diverse data. At the same time, many of the alive neurons are reserved for discrete features and act as token and n-gram detectors. Interestingly, their corresponding FFN updates not only promote next token candidates as could be expected, but also explicitly focus on removing the information about triggering them tokens, i.e., current input. To the best of our knowledge, this is the first example of mechanisms specialized at removing (rather than adding) information from the residual stream. With scale, models become more sparse in a sense that they have more dead neurons and token detectors. Finally, some neurons are positional: them being activated or not depends largely (or solely) on position and less so (or not at all) on textual data. We find that smaller models have sets of neurons acting as position range indicators while larger models operate in a less explicit manner.

大型語言模型中的神經元：死亡、N-gram、位置化

Neurons in Large Language Models: Dead, N-gram, Positional

摘要

Support