通过模块社区揭示大语言模型的认知模式

摘要

大型语言模型（LLMs）通过从科学发现、医疗诊断到聊天机器人等广泛应用，在科学、工程及社会领域带来了显著进步，重塑了我们的世界。尽管LLMs无处不在且实用，但其运作机制仍隐藏在数十亿参数与复杂结构之中，使得其内部架构与认知过程难以理解。针对这一空白，我们借鉴了生物学中理解新兴认知的方法，开发了一种基于网络的框架，将认知技能、LLM架构及数据集联系起来，从而引领了基础模型分析范式的转变。模块社区中的技能分布表明，尽管LLMs并未严格遵循特定生物系统中观察到的集中化专长模式，但它们展现出了独特的模块社区，其涌现的技能模式部分映射了鸟类和小型哺乳动物大脑中分布式却又相互关联的认知组织。我们的数值结果揭示了LLMs与生物系统间的一个关键差异：在LLMs中，技能获取极大地受益于动态的跨区域交互与神经可塑性。通过将认知科学原理与机器学习相结合，我们的框架为LLM的可解释性提供了新见解，并指出有效的微调策略应利用分布式学习动态，而非僵化的模块干预。

English

Large Language Models (LLMs) have reshaped our world with significant advancements in science, engineering, and society through applications ranging from scientific discoveries and medical diagnostics to Chatbots. Despite their ubiquity and utility, the underlying mechanisms of LLM remain concealed within billions of parameters and complex structures, making their inner architecture and cognitive processes challenging to comprehend. We address this gap by adopting approaches to understanding emerging cognition in biology and developing a network-based framework that links cognitive skills, LLM architectures, and datasets, ushering in a paradigm shift in foundation model analysis. The skill distribution in the module communities demonstrates that while LLMs do not strictly parallel the focalized specialization observed in specific biological systems, they exhibit unique communities of modules whose emergent skill patterns partially mirror the distributed yet interconnected cognitive organization seen in avian and small mammalian brains. Our numerical results highlight a key divergence from biological systems to LLMs, where skill acquisition benefits substantially from dynamic, cross-regional interactions and neural plasticity. By integrating cognitive science principles with machine learning, our framework provides new insights into LLM interpretability and suggests that effective fine-tuning strategies should leverage distributed learning dynamics rather than rigid modular interventions.

通过模块社区揭示大语言模型的认知模式

Unraveling the cognitive patterns of Large Language Models through module communities

摘要

Support