通過擴展高質量的指導對話來增強聊天語言模型

摘要

在指導數據上進行微調已被廣泛證實為實現像ChatGPT這樣的聊天語言模型的有效實踐。儘管這一點很直接，但擴大這些數據的多樣性和質量極有可能帶來性能的提升。本文旨在進一步提高開源模型的上限。我們首先提供了一個經過系統設計、多樣、信息豐富、大規模的指導對話數據集UltraChat，其中不涉及人類查詢。我們的目標是捕捉人類可能與AI助手互動的廣度，並採用全面的框架來迭代生成多輪對話。UltraChat 包含150萬個高質量的多輪對話，涵蓋了各種主題和指導。我們對UltraChat 的統計分析顯示其在各種關鍵指標上的優越性，包括規模、平均長度、多樣性、連貫性等，鞏固了其作為領先的開源數據集的地位。基於UltraChat，我們對LLaMA模型進行微調，創建了一個強大的對話模型UltraLLaMA。我們的評估表明，UltraLLaMA 在一致性地優於其他開源模型，包括之前公認的最先進的開源模型Vicuna。數據集和模型將被公開發布\url{https://github.com/thunlp/UltraChat}。

English

Fine-tuning on instruction data has been widely validated as an effective practice for implementing chat language models like ChatGPT. Scaling the diversity and quality of such data, although straightforward, stands a great chance of leading to improved performance. This paper aims to improve the upper bound of open-source models further. We first provide a systematically designed, diverse, informative, large-scale dataset of instructional conversations, UltraChat, which does not involve human queries. Our objective is to capture the breadth of interactions that a human might have with an AI assistant and employs a comprehensive framework to generate multi-turn conversation iteratively. UltraChat contains 1.5 million high-quality multi-turn dialogues and covers a wide range of topics and instructions. Our statistical analysis of UltraChat reveals its superiority in various key metrics, including scale, average length, diversity, coherence, etc., solidifying its position as a leading open-source dataset. Building upon UltraChat, we fine-tune a LLaMA model to create a powerful conversational model, UltraLLaMA. Our evaluations indicate that UltraLLaMA consistently outperforms other open-source models, including Vicuna, the previously recognized state-of-the-art open-source model. The dataset and the model will be publicly released\url{https://github.com/thunlp/UltraChat}.

通過擴展高質量的指導對話來增強聊天語言模型

Enhancing Chat Language Models by Scaling High-quality Instructional Conversations

摘要

Support