GeRe：通過通用樣本重播實現大型語言模型持續學習中的高效抗遺忘

GeRe: Towards Efficient Anti-Forgetting in Continual Learning of LLM via General Samples Replay

August 6, 2025

作者: Yunan Zhang, Shuoran Jiang, Mengchen Zhao, Yuefeng Li, Yang Fan, Xiangping Wu, Qingcai Chen

cs.AI

摘要

大型語言模型（LLMs）的持續學習能力對於推進人工通用智能至關重要。然而，跨多個領域對LLMs進行持續微調往往會遭遇災難性遺忘，其特徵表現為：1）對其通用能力的顯著遺忘，以及2）在先前學習任務中的性能急劇下降。為以簡單而穩定的方式同時解決這兩個問題，我們提出了通用樣本回放（GeRe）框架，該框架利用常規的預訓練文本來實現高效的反遺忘。在GeRe框架下重新審視最普遍的回放實踐之外，我們進一步利用神經狀態引入了一種基於閾值邊際（TM）損失的增強激活狀態約束優化方法，該方法在回放學習過程中保持激活狀態的一致性。我們首次驗證，一小組固定且預先收集的通用回放樣本足以解決這兩個問題——既保留通用能力，又提升在序列任務中的整體性能。事實上，前者本質上能夠促進後者。通過對照實驗，我們系統地比較了GeRe框架下TM與不同回放策略的性能，包括基於標籤擬合的原始方法、通過KL散度進行的logit模仿以及通過L1/L2損失進行的特徵模仿。結果表明，TM持續提升性能並展現出更好的魯棒性。我們的工作為未來LLMs的高效回放鋪平了道路。我們的代碼和數據可在https://github.com/Qznan/GeRe獲取。

English

The continual learning capability of large language models (LLMs) is crucial for advancing artificial general intelligence. However, continual fine-tuning LLMs across various domains often suffers from catastrophic forgetting, characterized by: 1) significant forgetting of their general capabilities, and 2) sharp performance declines in previously learned tasks. To simultaneously address both issues in a simple yet stable manner, we propose General Sample Replay (GeRe), a framework that use usual pretraining texts for efficient anti-forgetting. Beyond revisiting the most prevalent replay-based practices under GeRe, we further leverage neural states to introduce a enhanced activation states constrained optimization method using threshold-based margin (TM) loss, which maintains activation state consistency during replay learning. We are the first to validate that a small, fixed set of pre-collected general replay samples is sufficient to resolve both concerns--retaining general capabilities while promoting overall performance across sequential tasks. Indeed, the former can inherently facilitate the latter. Through controlled experiments, we systematically compare TM with different replay strategies under the GeRe framework, including vanilla label fitting, logit imitation via KL divergence and feature imitation via L1/L2 losses. Results demonstrate that TM consistently improves performance and exhibits better robustness. Our work paves the way for efficient replay of LLMs for the future. Our code and data are available at https://github.com/Qznan/GeRe.