符號學習使自我演化的智能體成為可能。

Symbolic Learning Enables Self-Evolving Agents

June 26, 2024

作者: Wangchunshu Zhou, Yixin Ou, Shengwei Ding, Long Li, Jialong Wu, Tiannan Wang, Jiamin Chen, Shuai Wang, Xiaohua Xu, Ningyu Zhang, Huajun Chen, Yuchen Eleanor Jiang

cs.AI

摘要

AI 社群一直在探索通往人工通用智能 (AGI) 的途徑，通過開發「語言代理」，這是複雜的大型語言模型 (LLMs) 管道，涉及提示技術和工具使用方法。雖然語言代理展示了在許多現實任務中的令人印象深刻的能力，但目前語言代理研究的一個基本限制是它們是以模型為中心，或者以工程為中心。也就是說，語言代理的提示、工具和管道的進展需要來自人類專家的大量手動工程努力，而不是自動從數據中學習。我們認為從以模型為中心或以工程為中心轉向以數據為中心，即語言代理能夠自主學習並在環境中進化的能力，是它們可能實現 AGI 的關鍵。 在本研究中，我們介紹了代理符號學習，這是一個系統性框架，使語言代理能夠以數據為中心的方式使用符號優化器自行優化自己。具體來說，我們將代理視為符號網絡，其中可學習的權重由提示、工具和它們如何堆疊來定義。代理符號學習旨在通過模仿聯結學習中的兩個基本算法：反向傳播和梯度下降，來優化語言代理內的符號網絡。代理符號學習不是處理數值權重，而是使用權重、損失和梯度的自然語言模擬。我們在標準基準和複雜現實任務上進行概念驗證實驗，結果顯示代理符號學習使語言代理能夠在創建並部署後更新自己，在野外演化，形成「自我演化代理」。

English

The AI community has been exploring a pathway to artificial general intelligence (AGI) by developing "language agents", which are complex large language models (LLMs) pipelines involving both prompting techniques and tool usage methods. While language agents have demonstrated impressive capabilities for many real-world tasks, a fundamental limitation of current language agents research is that they are model-centric, or engineering-centric. That's to say, the progress on prompts, tools, and pipelines of language agents requires substantial manual engineering efforts from human experts rather than automatically learning from data. We believe the transition from model-centric, or engineering-centric, to data-centric, i.e., the ability of language agents to autonomously learn and evolve in environments, is the key for them to possibly achieve AGI. In this work, we introduce agent symbolic learning, a systematic framework that enables language agents to optimize themselves on their own in a data-centric way using symbolic optimizers. Specifically, we consider agents as symbolic networks where learnable weights are defined by prompts, tools, and the way they are stacked together. Agent symbolic learning is designed to optimize the symbolic network within language agents by mimicking two fundamental algorithms in connectionist learning: back-propagation and gradient descent. Instead of dealing with numeric weights, agent symbolic learning works with natural language simulacrums of weights, loss, and gradients. We conduct proof-of-concept experiments on both standard benchmarks and complex real-world tasks and show that agent symbolic learning enables language agents to update themselves after being created and deployed in the wild, resulting in "self-evolving agents".