Trans-LoRA：朝向無需數據的可轉移參數高效微調前進。

Trans-LoRA: towards data-free Transferable Parameter Efficient Finetuning

May 27, 2024

作者: Runqian Wang, Soumya Ghosh, David Cox, Diego Antognini, Aude Oliva, Rogerio Feris, Leonid Karlinsky

cs.AI

摘要

低秩適配器（LoRA）及其變體是受歡迎的參數高效微調（PEFT）技術，可以與完整模型微調性能密切匹配，同時只需要少量額外參數。這些額外的LoRA參數是特定於正在適應的基礎模型的。當基礎模型需要被淘汰並替換為新模型時，所有相關的LoRA模塊都需要重新訓練。這種重新訓練需要訪問用於訓練原始基礎模型LoRA的數據。這對於商業雲應用尤其棘手，因為LoRA模塊和基礎模型由服務提供商托管，可能不允許托管專有客戶任務數據。為了應對這一挑戰，我們提出了Trans-LoRA——一種新穎的方法，可實現基於模型之間的LoRA無損、幾乎無需數據的轉移。我們的方法依賴於合成數據來轉移LoRA模塊。利用大型語言模型，我們設計了一個合成數據生成器，以近似觀察任務數據子集的生成過程。在生成的合成數據集上訓練，將LoRA模塊轉移到新模型。我們展示了我們的方法在LLama和Gemma模型系列上的有效性。我們的方法實現了在各種任務上模型內部和跨不同基礎模型系列之間，甚至在不同PEFT方法之間的LoRA轉移的無損（大多數情況下改進）。

English

Low-rank adapters (LoRA) and their variants are popular parameter-efficient fine-tuning (PEFT) techniques that closely match full model fine-tune performance while requiring only a small number of additional parameters. These additional LoRA parameters are specific to the base model being adapted. When the base model needs to be deprecated and replaced with a new one, all the associated LoRA modules need to be re-trained. Such re-training requires access to the data used to train the LoRA for the original base model. This is especially problematic for commercial cloud applications where the LoRA modules and the base models are hosted by service providers who may not be allowed to host proprietary client task data. To address this challenge, we propose Trans-LoRA -- a novel method for lossless, nearly data-free transfer of LoRAs across base models. Our approach relies on synthetic data to transfer LoRA modules. Using large language models, we design a synthetic data generator to approximate the data-generating process of the observed task data subset. Training on the resulting synthetic dataset transfers LoRA modules to new models. We show the effectiveness of our approach using both LLama and Gemma model families. Our approach achieves lossless (mostly improved) LoRA transfer between models within and across different base model families, and even between different PEFT methods, on a wide variety of tasks.