LLM代理操作系统

LLM Agent Operating System

March 25, 2024

作者: Kai Mei, Zelong Li, Shuyuan Xu, Ruosong Ye, Yingqiang Ge, Yongfeng Zhang

cs.AI

摘要

基于大型语言模型（LLM）的智能代理的集成和部署存在一系列挑战，这些挑战影响了它们的效率和功效。其中问题包括代理请求在LLM上的次优调度和资源分配、在代理和LLM之间交互时保持上下文的困难，以及整合具有不同能力和专业化的异构代理所固有的复杂性。代理数量和复杂性的快速增加进一步加剧了这些问题，通常导致资源瓶颈和资源利用的次优化。受到这些挑战的启发，本文提出了AIOS，即一个嵌入大型语言模型到操作系统（OS）中的LLM代理操作系统。具体来说，AIOS旨在优化资源分配、促进代理之间的上下文切换、实现代理的并发执行、为代理提供工具服务，并维护代理的访问控制。我们介绍了这种操作系统的架构，概述了它旨在解决的核心挑战，并提供了AIOS的基本设计和实现。我们对多个代理的并发执行进行的实验表明了我们AIOS模块的可靠性和效率。通过这一工作，我们旨在不仅提高LLM代理的性能和效率，还为未来更好地发展和部署AIOS生态系统开创先河。该项目在https://github.com/agiresearch/AIOS 上开源。

English

The integration and deployment of large language model (LLM)-based intelligent agents have been fraught with challenges that compromise their efficiency and efficacy. Among these issues are sub-optimal scheduling and resource allocation of agent requests over the LLM, the difficulties in maintaining context during interactions between agent and LLM, and the complexities inherent in integrating heterogeneous agents with different capabilities and specializations. The rapid increase of agent quantity and complexity further exacerbates these issues, often leading to bottlenecks and sub-optimal utilization of resources. Inspired by these challenges, this paper presents AIOS, an LLM agent operating system, which embeds large language model into operating systems (OS). Specifically, AIOS is designed to optimize resource allocation, facilitate context switch across agents, enable concurrent execution of agents, provide tool service for agents, and maintain access control for agents. We present the architecture of such an operating system, outline the core challenges it aims to resolve, and provide the basic design and implementation of the AIOS. Our experiments on concurrent execution of multiple agents demonstrate the reliability and efficiency of our AIOS modules. Through this, we aim to not only improve the performance and efficiency of LLM agents but also to pioneer for better development and deployment of the AIOS ecosystem in the future. The project is open-source at https://github.com/agiresearch/AIOS.