DataDreamer：一种用于合成数据生成和可重现LLM工作流的工具

DataDreamer: A Tool for Synthetic Data Generation and Reproducible LLM Workflows

February 16, 2024

作者: Ajay Patel, Colin Raffel, Chris Callison-Burch

cs.AI

摘要

大型语言模型（LLMs）已经成为自然语言处理研究人员在各种任务中的主要且重要工具。如今，许多研究人员在合成数据生成、任务评估、微调、蒸馏以及其他模型内部研究工作流程中使用LLMs。然而，使用这些模型时会遇到一些挑战，这些挑战源自它们的规模、封闭源特性以及缺乏针对这些新兴工作流程的标准化工具。这些模型迅速崭露头角以及这些独特挑战的出现立即对开放科学和使用它们的工作的可重复性产生了不利影响。在本文中，我们介绍了DataDreamer，这是一个开源的Python库，允许研究人员编写简单的代码来实现强大的LLM工作流程。DataDreamer还帮助研究人员遵循我们提出的最佳实践，以促进开放科学和可重复性。该库和文档可在 https://github.com/datadreamer-dev/DataDreamer 获取。

English

Large language models (LLMs) have become a dominant and important tool for NLP researchers in a wide range of tasks. Today, many researchers use LLMs in synthetic data generation, task evaluation, fine-tuning, distillation, and other model-in-the-loop research workflows. However, challenges arise when using these models that stem from their scale, their closed source nature, and the lack of standardized tooling for these new and emerging workflows. The rapid rise to prominence of these models and these unique challenges has had immediate adverse impacts on open science and on the reproducibility of work that uses them. In this paper, we introduce DataDreamer, an open source Python library that allows researchers to write simple code to implement powerful LLM workflows. DataDreamer also helps researchers adhere to best practices that we propose to encourage open science and reproducibility. The library and documentation are available at https://github.com/datadreamer-dev/DataDreamer .