關於大型語言模型中的關係特定神經元

On Relation-Specific Neurons in Large Language Models

February 24, 2025

作者: Yihong Liu, Runsheng Chen, Lea Hirlimann, Ahmad Dawar Hakimi, Mingyang Wang, Amir Hossein Kargaran, Sascha Rothe, François Yvon, Hinrich Schütze

cs.AI

摘要

在大型語言模型（LLMs）中，某些神經元可以存儲在預訓練期間學習到的不同知識片段。儘管知識通常呈現為關係和實體的組合，但目前尚不清楚是否有些神經元專注於關係本身 -- 與任何實體無關。我們假設這樣的神經元可以檢測輸入文本中的關係並引導涉及該關係的生成。為了探究這一點，我們使用基於統計的方法對選定的一組關係研究了 Llama-2 家族。我們的實驗證明了關係特定神經元的存在。我們測量了有針對性地停用與關係 r 相關的候選神經元對 LLMS 處理以下兩類事實的能力的影響：（1）其關係為 r 的事實和（2）其關係為不同關係 r'（r 不等於 r'）的事實。就其編碼關係信息的能力而言，我們提供了關於關係特定神經元以下三個特性的證據：（i）神經元累積性。與 r 相關的神經元呈現出累積效應，因此停用其中較大部分將導致 r 中更多事實的退化。（ii）神經元多功能性。神經元可以跨多個密切相關和較不相關的關係進行共享。一些關係神經元可以跨越語言。 （iii）神經元干擾。停用與一個關係特定的神經元可以提高 LLMS 對其他關係事實的生成性能。我們將使我們的代碼公開可用於 https://github.com/cisnlp/relation-specific-neurons。

English

In large language models (LLMs), certain neurons can store distinct pieces of knowledge learned during pretraining. While knowledge typically appears as a combination of relations and entities, it remains unclear whether some neurons focus on a relation itself -- independent of any entity. We hypothesize such neurons detect a relation in the input text and guide generation involving such a relation. To investigate this, we study the Llama-2 family on a chosen set of relations with a statistics-based method. Our experiments demonstrate the existence of relation-specific neurons. We measure the effect of selectively deactivating candidate neurons specific to relation r on the LLM's ability to handle (1) facts whose relation is r and (2) facts whose relation is a different relation r' neq r. With respect to their capacity for encoding relation information, we give evidence for the following three properties of relation-specific neurons. (i) Neuron cumulativity. The neurons for r present a cumulative effect so that deactivating a larger portion of them results in the degradation of more facts in r. (ii) Neuron versatility. Neurons can be shared across multiple closely related as well as less related relations. Some relation neurons transfer across languages. (iii) Neuron interference. Deactivating neurons specific to one relation can improve LLM generation performance for facts of other relations. We will make our code publicly available at https://github.com/cisnlp/relation-specific-neurons.