通过扩展高质量的指导性对话来增强聊天语言模型

摘要

在指导数据上进行微调已被广泛验证为实现类似ChatGPT的聊天语言模型的有效实践。尽管直接增加数据的多样性和质量看似简单，但却有很大机会提高性能。本文旨在进一步提高开源模型的上限。我们首先提供了一个系统设计的、多样化的、信息丰富的大规模指导对话数据集UltraChat，其中不涉及人类查询。我们的目标是捕捉人类可能与AI助手进行的各种互动，并采用全面的框架迭代生成多轮对话。UltraChat包含150万条高质量的多轮对话，涵盖了广泛的主题和指导。我们对UltraChat的统计分析显示其在各种关键指标上的优越性，包括规模、平均长度、多样性、连贯性等，巩固了其作为领先的开源数据集的地位。基于UltraChat，我们对LLaMA模型进行微调，创建了一个强大的对话模型UltraLLaMA。我们的评估表明，UltraLLaMA在性能上始终优于其他开源模型，包括之前公认的最先进的开源模型Vicuna。该数据集和模型将被公开发布\url{https://github.com/thunlp/UltraChat}。

English

Fine-tuning on instruction data has been widely validated as an effective practice for implementing chat language models like ChatGPT. Scaling the diversity and quality of such data, although straightforward, stands a great chance of leading to improved performance. This paper aims to improve the upper bound of open-source models further. We first provide a systematically designed, diverse, informative, large-scale dataset of instructional conversations, UltraChat, which does not involve human queries. Our objective is to capture the breadth of interactions that a human might have with an AI assistant and employs a comprehensive framework to generate multi-turn conversation iteratively. UltraChat contains 1.5 million high-quality multi-turn dialogues and covers a wide range of topics and instructions. Our statistical analysis of UltraChat reveals its superiority in various key metrics, including scale, average length, diversity, coherence, etc., solidifying its position as a leading open-source dataset. Building upon UltraChat, we fine-tune a LLaMA model to create a powerful conversational model, UltraLLaMA. Our evaluations indicate that UltraLLaMA consistently outperforms other open-source models, including Vicuna, the previously recognized state-of-the-art open-source model. The dataset and the model will be publicly released\url{https://github.com/thunlp/UltraChat}.

通过扩展高质量的指导性对话来增强聊天语言模型

Enhancing Chat Language Models by Scaling High-quality Instructional Conversations

摘要

Support