Trans-LoRA：走向无数据迁移参数高效微调

Trans-LoRA: towards data-free Transferable Parameter Efficient Finetuning

May 27, 2024

作者: Runqian Wang, Soumya Ghosh, David Cox, Diego Antognini, Aude Oliva, Rogerio Feris, Leonid Karlinsky

cs.AI

摘要

低秩适配器（LoRA）及其变体是流行的参数高效微调（PEFT）技术，可以在仅需要少量额外参数的情况下实现与完整模型微调性能相近。这些额外的LoRA参数是特定于正在适配的基础模型的。当需要废弃基础模型并用新模型替换时，所有相关的LoRA模块都需要重新训练。这种重新训练需要访问用于训练原始基础模型LoRA的数据。这对于商业云应用尤其具有问题，因为LoRA模块和基础模型由服务提供商托管，可能不允许托管专有客户任务数据。为解决这一挑战，我们提出Trans-LoRA -- 一种新颖的方法，可在基础模型之间实现无损、几乎无需数据的LoRA转移。我们的方法依赖于合成数据来转移LoRA模块。利用大型语言模型，我们设计了一个合成数据生成器，以近似观察任务数据子集的生成过程。在生成的合成数据集上训练，将LoRA模块转移到新模型。我们展示了我们的方法在LLama和Gemma模型系列上的有效性。我们的方法在各种任务上实现了在模型内部和跨不同基础模型系列之间，甚至在不同PEFT方法之间的无损（大多数改进）LoRA转移。

English

Low-rank adapters (LoRA) and their variants are popular parameter-efficient fine-tuning (PEFT) techniques that closely match full model fine-tune performance while requiring only a small number of additional parameters. These additional LoRA parameters are specific to the base model being adapted. When the base model needs to be deprecated and replaced with a new one, all the associated LoRA modules need to be re-trained. Such re-training requires access to the data used to train the LoRA for the original base model. This is especially problematic for commercial cloud applications where the LoRA modules and the base models are hosted by service providers who may not be allowed to host proprietary client task data. To address this challenge, we propose Trans-LoRA -- a novel method for lossless, nearly data-free transfer of LoRAs across base models. Our approach relies on synthetic data to transfer LoRA modules. Using large language models, we design a synthetic data generator to approximate the data-generating process of the observed task data subset. Training on the resulting synthetic dataset transfers LoRA modules to new models. We show the effectiveness of our approach using both LLama and Gemma model families. Our approach achieves lossless (mostly improved) LoRA transfer between models within and across different base model families, and even between different PEFT methods, on a wide variety of tasks.