LLM 代理操作系统

LLM Agent Operating System

March 25, 2024

作者: Kai Mei, Zelong Li, Shuyuan Xu, Ruosong Ye, Yingqiang Ge, Yongfeng Zhang

cs.AI

摘要

基於大型語言模型（LLM）的智能代理的整合和部署一直面臨著挑戰，這些挑戰影響了它們的效率和功效。其中問題包括代理請求在LLM上的次優排程和資源分配、在代理和LLM之間互動時維持上下文的困難，以及整合具有不同能力和專業的異質代理所固有的複雜性。代理數量和複雜性的快速增加進一步惡化了這些問題，通常導致瓶頸和資源的次優利用。受到這些挑戰的啟發，本文提出了AIOS，一個LLM代理操作系統，將大型語言模型嵌入操作系統（OS）中。具體而言，AIOS旨在優化資源分配、促進代理之間的上下文切換、實現代理的並行執行、為代理提供工具服務，以及維護代理的訪問控制。我們介紹了這種操作系統的架構，概述了它旨在解決的核心挑戰，並提供了AIOS的基本設計和實現。我們對多個代理的並行執行進行的實驗證明了我們AIOS模組的可靠性和效率。通過這一工作，我們旨在不僅提高LLM代理的性能和效率，還為未來AIOS生態系統的更好開發和部署開創先河。該項目在https://github.com/agiresearch/AIOS 上以開源方式提供。

English

The integration and deployment of large language model (LLM)-based intelligent agents have been fraught with challenges that compromise their efficiency and efficacy. Among these issues are sub-optimal scheduling and resource allocation of agent requests over the LLM, the difficulties in maintaining context during interactions between agent and LLM, and the complexities inherent in integrating heterogeneous agents with different capabilities and specializations. The rapid increase of agent quantity and complexity further exacerbates these issues, often leading to bottlenecks and sub-optimal utilization of resources. Inspired by these challenges, this paper presents AIOS, an LLM agent operating system, which embeds large language model into operating systems (OS). Specifically, AIOS is designed to optimize resource allocation, facilitate context switch across agents, enable concurrent execution of agents, provide tool service for agents, and maintain access control for agents. We present the architecture of such an operating system, outline the core challenges it aims to resolve, and provide the basic design and implementation of the AIOS. Our experiments on concurrent execution of multiple agents demonstrate the reliability and efficiency of our AIOS modules. Through this, we aim to not only improve the performance and efficiency of LLM agents but also to pioneer for better development and deployment of the AIOS ecosystem in the future. The project is open-source at https://github.com/agiresearch/AIOS.