当扩展遇上LLM微调：数据、模型和微调方法的影响

摘要

尽管大型语言模型（LLMs）通常采用微调来释放其在下游应用中的能力，但我们对不同微调方法的归纳偏差（尤其是尺度特性）的理解仍然有限。为了填补这一空白，我们进行了系统实验，研究不同尺度因素（包括LLM模型大小、预训练数据大小、新微调参数大小和微调数据大小）如何影响微调性能。我们考虑了两种微调类型--全模型微调（FMT）和参数高效微调（PET，包括提示微调和LoRA），并探讨它们在数据有限情况下的尺度行为，其中LLM模型大小远远超过微调数据大小。基于两组预训练的双语LLMs（从1B到16B）以及在双语机器翻译和多语言摘要基准测试上的实验，我们发现：1）LLM微调遵循一种基于乘法的幂律联合缩放规律，介于微调数据大小和其他尺度因素之间；2）LLM微调更多地受益于LLM模型尺度而不是预训练数据尺度，PET参数尺度通常无效；3）最佳微调方法高度依赖于任务和微调数据。我们希望我们的发现能够帮助理解、选择和发展LLM微调方法。

English

While large language models (LLMs) often adopt finetuning to unlock their capabilities for downstream applications, our understanding on the inductive biases (especially the scaling properties) of different finetuning methods is still limited. To fill this gap, we conduct systematic experiments studying whether and how different scaling factors, including LLM model size, pretraining data size, new finetuning parameter size and finetuning data size, affect the finetuning performance. We consider two types of finetuning -- full-model tuning (FMT) and parameter efficient tuning (PET, including prompt tuning and LoRA), and explore their scaling behaviors in the data-limited regime where the LLM model size substantially outweighs the finetuning data size. Based on two sets of pretrained bilingual LLMs from 1B to 16B and experiments on bilingual machine translation and multilingual summarization benchmarks, we find that 1) LLM finetuning follows a powerbased multiplicative joint scaling law between finetuning data size and each other scaling factor; 2) LLM finetuning benefits more from LLM model scaling than pretraining data scaling, and PET parameter scaling is generally ineffective; and 3) the optimal finetuning method is highly task- and finetuning data-dependent. We hope our findings could shed light on understanding, selecting and developing LLM finetuning methods.

当扩展遇上LLM微调：数据、模型和微调方法的影响

When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method

摘要

Support