CURLoRA：稳定的LLM持续微调和灾难性遗忘缓解

CURLoRA: Stable LLM Continual Fine-Tuning and Catastrophic Forgetting Mitigation

August 26, 2024

作者: Muhammad Fawi

cs.AI

摘要

本文介绍了CURLoRA，这是一种新颖的方法，用于在大型语言模型（LLMs）中利用CUR矩阵分解来进行微调，结合了低秩适应（LoRA）的概念。我们的方法解决了LLM微调中的两个关键挑战：在持续学习过程中减轻灾难性遗忘，并减少可训练参数的数量。我们对CUR分解过程进行了独特修改，利用反转概率进行列和行的选择，作为一种隐式正则化，并将U矩阵初始化为零矩阵，仅对其进行微调。通过在多个数据集上进行实验证明，CURLoRA在减轻灾难性遗忘方面优于标准LoRA。它在各项任务中保持模型稳定性和性能，同时显著减少可训练参数的数量。我们的结果表明，与LoRA相比，在持续微调过程中，尤其是在数据有限的情况下，CURLoRA实现了非常好的和稳定的任务准确性，同时保持基础模型的困惑度分数不变。

English

This paper introduces CURLoRA, a novel approach to fine-tuning large language models (LLMs) that leverages CUR matrix decomposition in the context of Low-Rank Adaptation (LoRA). Our method addresses two critical challenges in LLM fine-tuning: mitigating catastrophic forgetting during continual learning and reducing the number of trainable parameters. We propose a unique modification to the CUR decomposition process, utilizing inverted probabilities for column and row selection which acts as an implicit regularization, and initializing the U matrix as a zero matrix, and only fine-tuning it. We demonstrate through experiments on multiple datasets that CURLoRA outperforms standard LoRA in mitigating catastrophic forgetting. It maintains model stability and performance across tasks while significantly reducing the number of trainable parameters. Our results show that CURLoRA achieves very good and stable task accuracy while maintaining base model's perplexity scores fixed compared to LoRA upon continual fine-tuning, particularly in scenarios with limited data.