MFTCoder：利用多任务微调提升代码LLMs

MFTCoder: Boosting Code LLMs with Multitask Fine-Tuning

November 4, 2023

作者: Bingchang Liu, Chaoyu Chen, Cong Liao, Zi Gong, Huan Wang, Zhichao Lei, Ming Liang, Dajun Chen, Min Shen, Hailian Zhou, Hang Yu, Jianguo Li

cs.AI

摘要

代码LLMs已经成为一个专门的研究领域，有着显著的研究致力于通过对预训练模型进行微调来增强模型的编码能力。先前的微调方法通常针对特定的下游任务或场景进行定制，这意味着针对每个任务单独进行微调，需要大量的训练资源，并在部署和维护方面存在挑战。此外，这些方法未能利用不同与代码相关任务之间的内在相互关联性。为了克服这些限制，我们提出了一个多任务微调框架，MFTcoder，它可以实现对多个任务进行同时和并行微调。通过结合各种损失函数，我们有效地解决了多任务学习中的常见挑战，如数据不平衡、不同的难度级别和不一致的收敛速度。大量实验证明，我们的多任务微调方法明显优于单个任务的个别微调和混合任务集的微调。此外，MFTcoder提供了高效的训练能力，包括高效的数据标记模式和PEFT微调，相比传统的微调方法，速度显著提高。MFTcoder与几个主流开源LLMs（如CodeLLama和Qwen）无缝集成。利用CodeLLama基础，我们的MFTcoder微调模型CodeFuse-CodeLLama-34B在HumaneEval基准测试中取得了令人印象深刻的74.4\%的pass@1分数，超过了GPT-4的性能（67\%，零-shot）。MFTCoder在https://github.com/codefuse-ai/MFTCOder上开源。

English

Code LLMs have emerged as a specialized research field, with remarkable studies dedicated to enhancing model's coding capabilities through fine-tuning on pre-trained models. Previous fine-tuning approaches were typically tailored to specific downstream tasks or scenarios, which meant separate fine-tuning for each task, requiring extensive training resources and posing challenges in terms of deployment and maintenance. Furthermore, these approaches failed to leverage the inherent interconnectedness among different code-related tasks. To overcome these limitations, we present a multi-task fine-tuning framework, MFTcoder, that enables simultaneous and parallel fine-tuning on multiple tasks. By incorporating various loss functions, we effectively address common challenges in multi-task learning, such as data imbalance, varying difficulty levels, and inconsistent convergence speeds. Extensive experiments have conclusively demonstrated that our multi-task fine-tuning approach outperforms both individual fine-tuning on single tasks and fine-tuning on a mixed ensemble of tasks. Moreover, MFTcoder offers efficient training capabilities, including efficient data tokenization modes and PEFT fine-tuning, resulting in significantly improved speed compared to traditional fine-tuning methods. MFTcoder seamlessly integrates with several mainstream open-source LLMs, such as CodeLLama and Qwen. Leveraging the CodeLLama foundation, our MFTcoder fine-tuned model, CodeFuse-CodeLLama-34B, achieves an impressive pass@1 score of 74.4\% on the HumaneEval benchmark, surpassing GPT-4 performance (67\%, zero-shot). MFTCoder is open-sourced at https://github.com/codefuse-ai/MFTCOder