DSPy：将声明性语言模型调用编译为自我改进的管道

DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines

October 5, 2023

作者: Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Sri Vardhamanan, Saiful Haq, Ashutosh Sharma, Thomas T. Joshi, Hanna Moazam, Heather Miller, Matei Zaharia, Christopher Potts

cs.AI

摘要

机器学习社区正在迅速探索提示语言模型（LMs）和将它们堆叠成解决复杂任务的管道的技术。不幸的是，现有的LM管道通常是使用硬编码的“提示模板”实现的，即通过试错发现的冗长字符串。为了更系统地开发和优化LM管道，我们引入了DSPy，这是一个将LM管道抽象为文本转换图的编程模型，即命令式计算图，其中LMs通过声明性模块调用。DSPy模块是参数化的，这意味着它们可以学习（通过创建和收集演示）如何应用提示、微调、增强和推理技术的组合。我们设计了一个编译器，将优化任何DSPy管道以最大化给定的度量。我们进行了两个案例研究，展示简洁的DSPy程序可以表达和优化复杂的LM管道，用于解决数学问题、处理多跳检索、回答复杂问题和控制代理循环。在几分钟的编译后，几行DSPy代码就可以让GPT-3.5和llama2-13b-chat自助引导管道，胜过标准的少样本提示（一般分别高出25%和65%），以及具有专家创建演示的管道（分别高出5-46%和16-40%）。此外，编译为开放和相对较小的LMs（如770M参数的T5和llama2-13b-chat）的DSPy程序与依赖专家编写的提示链的专有GPT-3.5方法相竞争。DSPy可在https://github.com/stanfordnlp/dspy获得。

English

The ML community is rapidly exploring techniques for prompting language models (LMs) and for stacking them into pipelines that solve complex tasks. Unfortunately, existing LM pipelines are typically implemented using hard-coded "prompt templates", i.e. lengthy strings discovered via trial and error. Toward a more systematic approach for developing and optimizing LM pipelines, we introduce DSPy, a programming model that abstracts LM pipelines as text transformation graphs, i.e. imperative computational graphs where LMs are invoked through declarative modules. DSPy modules are parameterized, meaning they can learn (by creating and collecting demonstrations) how to apply compositions of prompting, finetuning, augmentation, and reasoning techniques. We design a compiler that will optimize any DSPy pipeline to maximize a given metric. We conduct two case studies, showing that succinct DSPy programs can express and optimize sophisticated LM pipelines that reason about math word problems, tackle multi-hop retrieval, answer complex questions, and control agent loops. Within minutes of compiling, a few lines of DSPy allow GPT-3.5 and llama2-13b-chat to self-bootstrap pipelines that outperform standard few-shot prompting (generally by over 25% and 65%, respectively) and pipelines with expert-created demonstrations (by up to 5-46% and 16-40%, respectively). On top of that, DSPy programs compiled to open and relatively small LMs like 770M-parameter T5 and llama2-13b-chat are competitive with approaches that rely on expert-written prompt chains for proprietary GPT-3.5. DSPy is available at https://github.com/stanfordnlp/dspy