CURLoRA：穩定LLM持續微調與災難性遺忘緩解

CURLoRA: Stable LLM Continual Fine-Tuning and Catastrophic Forgetting Mitigation

August 26, 2024

作者: Muhammad Fawi

cs.AI

摘要

本文介紹了一種名為CURLoRA的新方法，用於在大型語言模型（LLMs）上進行微調，該方法利用CUR矩陣分解在低秩適應（LoRA）的背景下。我們的方法解決了LLM微調中的兩個關鍵挑戰：在持續學習期間減輕災難性遺忘，並減少可訓練參數的數量。我們對CUR分解過程進行了獨特修改，利用反轉概率進行列和行的選擇，作為一種隱式正則化，並將U矩陣初始化為零矩陣，僅進行微調。通過對多個數據集進行實驗，我們證明了CURLoRA在減輕災難性遺忘方面優於標準LoRA。它在各任務之間保持模型穩定性和性能，同時顯著減少可訓練參數的數量。我們的結果表明，與LoRA相比，在持續微調時，尤其是在數據有限的情況下，CURLoRA實現了非常好且穩定的任務準確性，同時保持基本模型的困惑度分數不變。

English

This paper introduces CURLoRA, a novel approach to fine-tuning large language models (LLMs) that leverages CUR matrix decomposition in the context of Low-Rank Adaptation (LoRA). Our method addresses two critical challenges in LLM fine-tuning: mitigating catastrophic forgetting during continual learning and reducing the number of trainable parameters. We propose a unique modification to the CUR decomposition process, utilizing inverted probabilities for column and row selection which acts as an implicit regularization, and initializing the U matrix as a zero matrix, and only fine-tuning it. We demonstrate through experiments on multiple datasets that CURLoRA outperforms standard LoRA in mitigating catastrophic forgetting. It maintains model stability and performance across tasks while significantly reducing the number of trainable parameters. Our results show that CURLoRA achieves very good and stable task accuracy while maintaining base model's perplexity scores fixed compared to LoRA upon continual fine-tuning, particularly in scenarios with limited data.