基于LLM的复合人工智能系统优化：调研

LLM-based Optimization of Compound AI Systems: A Survey

October 21, 2024

作者: Matthieu Lin, Jenny Sheng, Andrew Zhao, Shenzhi Wang, Yang Yue, Yiran Wu, Huan Liu, Jun Liu, Gao Huang, Yong-Jin Liu

cs.AI

摘要

在一个复合人工智能系统中，诸如LLM调用、检索器、代码解释器或工具等组件是相互连接的。系统的行为主要由诸如指令或工具定义之类的参数驱动。最近的进展使得能够利用LLM对这些参数进行端到端优化。值得注意的是，利用LLM作为优化器特别高效，因为它避免了梯度计算，并且能够生成复杂的代码和指令。本文介绍了基于LLM对复合人工智能系统进行优化的原则和新兴趋势调查。它涵盖了复合人工智能系统的典型类型、基于LLM的端到端优化方法，以及对未来方向和更广泛影响的见解。重要的是，这项调查运用了程序分析的概念，以提供LLM优化器如何被促使优化复合人工智能系统的统一视角。有关详尽的论文列表，请访问https://github.com/linyuhongg/LLM-based-Optimization-of-Compound-AI-Systems。

English

In a compound AI system, components such as an LLM call, a retriever, a code interpreter, or tools are interconnected. The system's behavior is primarily driven by parameters such as instructions or tool definitions. Recent advancements enable end-to-end optimization of these parameters using an LLM. Notably, leveraging an LLM as an optimizer is particularly efficient because it avoids gradient computation and can generate complex code and instructions. This paper presents a survey of the principles and emerging trends in LLM-based optimization of compound AI systems. It covers archetypes of compound AI systems, approaches to LLM-based end-to-end optimization, and insights into future directions and broader impacts. Importantly, this survey uses concepts from program analysis to provide a unified view of how an LLM optimizer is prompted to optimize a compound AI system. The exhaustive list of paper is provided at https://github.com/linyuhongg/LLM-based-Optimization-of-Compound-AI-Systems.