透過模組社群解析大型語言模型的認知模式

摘要

大型語言模型（LLMs）透過從科學發現、醫學診斷到聊天機器人等廣泛應用，在科學、工程和社會領域帶來了顯著進步，重塑了我們的世界。儘管它們無處不在且實用性高，但LLM的運作機制仍隱藏在數十億參數和複雜結構之中，使其內部架構和認知過程難以理解。我們通過借鑒生物學中理解新興認知的方法，並開發一個基於網絡的框架來彌補這一差距，該框架將認知技能、LLM架構和數據集聯繫起來，從而引領基礎模型分析的範式轉變。模塊社群中的技能分佈表明，雖然LLMs並未嚴格對應於特定生物系統中觀察到的集中專化，但它們展現了獨特的模塊社群，其湧現的技能模式部分反映了鳥類和小型哺乳動物大腦中分佈式卻相互連接的認知組織。我們的數值結果突顯了從生物系統到LLMs的一個關鍵差異，即技能獲取在很大程度上受益於動態的跨區域互動和神經可塑性。通過將認知科學原理與機器學習相結合，我們的框架為LLM的可解釋性提供了新的見解，並表明有效的微調策略應利用分佈式學習動態，而非僵化的模塊干預。

English

Large Language Models (LLMs) have reshaped our world with significant advancements in science, engineering, and society through applications ranging from scientific discoveries and medical diagnostics to Chatbots. Despite their ubiquity and utility, the underlying mechanisms of LLM remain concealed within billions of parameters and complex structures, making their inner architecture and cognitive processes challenging to comprehend. We address this gap by adopting approaches to understanding emerging cognition in biology and developing a network-based framework that links cognitive skills, LLM architectures, and datasets, ushering in a paradigm shift in foundation model analysis. The skill distribution in the module communities demonstrates that while LLMs do not strictly parallel the focalized specialization observed in specific biological systems, they exhibit unique communities of modules whose emergent skill patterns partially mirror the distributed yet interconnected cognitive organization seen in avian and small mammalian brains. Our numerical results highlight a key divergence from biological systems to LLMs, where skill acquisition benefits substantially from dynamic, cross-regional interactions and neural plasticity. By integrating cognitive science principles with machine learning, our framework provides new insights into LLM interpretability and suggests that effective fine-tuning strategies should leverage distributed learning dynamics rather than rigid modular interventions.

透過模組社群解析大型語言模型的認知模式

Unraveling the cognitive patterns of Large Language Models through module communities

摘要

Support