符号学习实现自进化智能体

Symbolic Learning Enables Self-Evolving Agents

June 26, 2024

作者: Wangchunshu Zhou, Yixin Ou, Shengwei Ding, Long Li, Jialong Wu, Tiannan Wang, Jiamin Chen, Shuai Wang, Xiaohua Xu, Ningyu Zhang, Huajun Chen, Yuchen Eleanor Jiang

cs.AI

摘要

AI社区一直在探索通往人工通用智能（AGI）的途径，通过开发“语言代理”，这些代理是复杂的大型语言模型（LLMs）管道，涉及提示技术和工具使用方法。虽然语言代理在许多现实世界任务中展示了令人印象深刻的能力，但当前语言代理研究的一个基本限制是它们是以模型为中心或以工程为中心的。也就是说，语言代理的提示、工具和管道的进展需要人类专家进行大量手工工程工作，而不是自动从数据中学习。我们认为，从以模型为中心或以工程为中心转变为以数据为中心，即语言代理能够在环境中自主学习和演化的能力，是它们可能实现AGI的关键。 在这项工作中，我们介绍了代理符号学习，这是一个系统框架，使语言代理能够以数据为中心的方式使用符号优化器自我优化。具体而言，我们将代理视为符号网络，其中可学习的权重由提示、工具和它们的堆叠方式定义。代理符号学习旨在通过模仿连接主义学习中的两个基本算法：反向传播和梯度下降，优化语言代理内的符号网络。代理符号学习不处理数值权重，而是使用权重、损失和梯度的自然语言模拟。我们在标准基准和复杂的现实世界任务上进行了概念验证实验，并展示代理符号学习使语言代理能够在创建和部署后更新自身，从而产生“自我演化代理”。

English

The AI community has been exploring a pathway to artificial general intelligence (AGI) by developing "language agents", which are complex large language models (LLMs) pipelines involving both prompting techniques and tool usage methods. While language agents have demonstrated impressive capabilities for many real-world tasks, a fundamental limitation of current language agents research is that they are model-centric, or engineering-centric. That's to say, the progress on prompts, tools, and pipelines of language agents requires substantial manual engineering efforts from human experts rather than automatically learning from data. We believe the transition from model-centric, or engineering-centric, to data-centric, i.e., the ability of language agents to autonomously learn and evolve in environments, is the key for them to possibly achieve AGI. In this work, we introduce agent symbolic learning, a systematic framework that enables language agents to optimize themselves on their own in a data-centric way using symbolic optimizers. Specifically, we consider agents as symbolic networks where learnable weights are defined by prompts, tools, and the way they are stacked together. Agent symbolic learning is designed to optimize the symbolic network within language agents by mimicking two fundamental algorithms in connectionist learning: back-propagation and gradient descent. Instead of dealing with numeric weights, agent symbolic learning works with natural language simulacrums of weights, loss, and gradients. We conduct proof-of-concept experiments on both standard benchmarks and complex real-world tasks and show that agent symbolic learning enables language agents to update themselves after being created and deployed in the wild, resulting in "self-evolving agents".