WizardCoder：利用Evol-Instruct赋能编码大型语言模型

摘要

大型语言模型编码（Code LLMs），如StarCoder，在与代码相关的任务中表现出色。然而，大多数现有模型仅在广泛的原始代码数据上进行预训练，没有进行指导微调。本文介绍了WizardCoder，它通过将Evol-Instruct方法调整到代码领域，为Code LLMs提供了复杂的指导微调能力。通过在四个著名的代码生成基准上进行全面实验，即HumanEval、HumanEval+、MBPP和DS-1000，我们展示了我们模型的卓越能力。它在所有其他开源Code LLMs上取得了显著的优势。此外，我们的模型甚至在HumanEval和HumanEval+上超越了最大的封闭LLMs，Anthropic的Claude和Google的Bard。我们的代码、模型权重和数据可在https://github.com/nlpxucan/WizardLM 上公开获取。

English

Code Large Language Models (Code LLMs), such as StarCoder, have demonstrated exceptional performance in code-related tasks. However, most existing models are solely pre-trained on extensive raw code data without instruction fine-tuning. In this paper, we introduce WizardCoder, which empowers Code LLMs with complex instruction fine-tuning, by adapting the Evol-Instruct method to the domain of code. Through comprehensive experiments on four prominent code generation benchmarks, namely HumanEval, HumanEval+, MBPP, and DS-1000, we unveil the exceptional capabilities of our model. It surpasses all other open-source Code LLMs by a substantial margin. Moreover, our model even outperforms the largest closed LLMs, Anthropic's Claude and Google's Bard, on HumanEval and HumanEval+. Our code, model weights, and data are public at https://github.com/nlpxucan/WizardLM

WizardCoder：利用Evol-Instruct赋能编码大型语言模型

WizardCoder: Empowering Code Large Language Models with Evol-Instruct

摘要

Support