RECALL：基於分層模型合併的表徵對齊式災難性遺忘緩解技術

摘要

我們揭示大型語言模型（LLMs）的內部表徵可作為所學知識的可靠代理，並提出RECALL——一種無需歷史數據、基於表徵感知的持續學習模型融合新框架。該框架通過聚類典型樣本上的層級隱藏表徵計算模型間相似度，執行自適應的層次化參數融合以實現知識對齊。此設計既能保留淺層的領域通用特徵，又允許深層進行任務特定適配。有別於需任務標籤或犧牲性能的既有方法，RECALL實現了無縫多領域整合與強大的災難性遺忘抵抗能力。在五項NLP任務與多種持續學習場景下的廣泛實驗表明，RECALL在知識保留與泛化能力上均超越基線方法，為LLMs的演進提供了可擴展的無數據解決方案。

English

We unveil that internal representations in large language models (LLMs) serve as reliable proxies of learned knowledge, and propose RECALL, a novel representation-aware model merging framework for continual learning without access to historical data. RECALL computes inter-model similarity from layer-wise hidden representations over clustered typical samples, and performs adaptive, hierarchical parameter fusion to align knowledge across models. This design enables the preservation of domain-general features in shallow layers while allowing task-specific adaptation in deeper layers. Unlike prior methods that require task labels or incur performance trade-offs, RECALL achieves seamless multi-domain integration and strong resistance to catastrophic forgetting. Extensive experiments across five NLP tasks and multiple continual learning scenarios show that RECALL outperforms baselines in both knowledge retention and generalization, providing a scalable and data-free solution for evolving LLMs.

RECALL：基於分層模型合併的表徵對齊式災難性遺忘緩解技術

RECALL: REpresentation-aligned Catastrophic-forgetting ALLeviation via Hierarchical Model Merging

摘要

Support