基于表征对齐的分层模型融合灾难性遗忘缓解方法

摘要

我们揭示了大语言模型（LLMs）的内部表征可作为所学知识的可靠代理，并提出了RECALL——一种无需历史数据即可实现持续学习的表征感知模型融合新框架。该框架通过聚类典型样本上的分层隐藏表征计算模型间相似度，执行自适应的层次化参数融合以实现知识对齐。该设计能保留浅层的领域通用特征，同时允许深层进行任务特异性适配。与需要任务标签或牺牲性能的现有方法不同，RECALL实现了多领域无缝集成与强大的抗灾难性遗忘能力。在五个NLP任务和多种持续学习场景下的大规模实验表明，RECALL在知识保留与泛化能力上均超越基线方法，为LLMs的持续演进提供了可扩展的无数据解决方案。

English

We unveil that internal representations in large language models (LLMs) serve as reliable proxies of learned knowledge, and propose RECALL, a novel representation-aware model merging framework for continual learning without access to historical data. RECALL computes inter-model similarity from layer-wise hidden representations over clustered typical samples, and performs adaptive, hierarchical parameter fusion to align knowledge across models. This design enables the preservation of domain-general features in shallow layers while allowing task-specific adaptation in deeper layers. Unlike prior methods that require task labels or incur performance trade-offs, RECALL achieves seamless multi-domain integration and strong resistance to catastrophic forgetting. Extensive experiments across five NLP tasks and multiple continual learning scenarios show that RECALL outperforms baselines in both knowledge retention and generalization, providing a scalable and data-free solution for evolving LLMs.

基于表征对齐的分层模型融合灾难性遗忘缓解方法

RECALL: REpresentation-aligned Catastrophic-forgetting ALLeviation via Hierarchical Model Merging

摘要

Support