多智能體系統中的潛在協作

Latent Collaboration in Multi-Agent Systems

November 25, 2025

作者: Jiaru Zou, Xiyuan Yang, Ruizhong Qiu, Gaotang Li, Katherine Tieu, Pan Lu, Ke Shen, Hanghang Tong, Yejin Choi, Jingrui He, James Zou, Mengdi Wang, Ling Yang

cs.AI

摘要

多智能體系統（MAS）將大型語言模型（LLMs）從獨立的單模型推理擴展至協作式的系統級智能。現有LLM智能體依賴基於文本的介導進行推理與通信，我們更進一步實現模型在連續潛空間內的直接協作。本文提出LatentMAS——一種免訓練的端到端框架，實現LLM智能體間的純潛在協作。在LatentMAS中，每個智能體首先透過最後一層隱藏嵌入進行自迴歸潛在思維生成，隨後由共享潛在工作記憶保存並傳遞各智能體的內部表徵，確保無損信息交換。理論分析表明，相較傳統基於文本的MAS，LatentMAS能以顯著更低的複雜度實現更高表達力與無損信息保留。此外，在涵蓋數學科學推理、常識理解與代碼生成的9個綜合基準測試中，LatentMAS持續優化強力單模型及文本型MAS基線：準確率最高提升14.6%，輸出標記使用量減少70.8%-83.7%，端到端推理速度提升4-4.3倍。這些結果證明我們的新型潛在協作框架在提升系統級推理質量的同時，無需額外訓練即可實現顯著效率增益。代碼與數據已於https://github.com/Gen-Verse/LatentMAS 全面開源。

English

Multi-agent systems (MAS) extend large language models (LLMs) from independent single-model reasoning to coordinative system-level intelligence. While existing LLM agents depend on text-based mediation for reasoning and communication, we take a step forward by enabling models to collaborate directly within the continuous latent space. We introduce LatentMAS, an end-to-end training-free framework that enables pure latent collaboration among LLM agents. In LatentMAS, each agent first performs auto-regressive latent thoughts generation through last-layer hidden embeddings. A shared latent working memory then preserves and transfers each agent's internal representations, ensuring lossless information exchange. We provide theoretical analyses establishing that LatentMAS attains higher expressiveness and lossless information preservation with substantially lower complexity than vanilla text-based MAS. In addition, empirical evaluations across 9 comprehensive benchmarks spanning math and science reasoning, commonsense understanding, and code generation show that LatentMAS consistently outperforms strong single-model and text-based MAS baselines, achieving up to 14.6% higher accuracy, reducing output token usage by 70.8%-83.7%, and providing 4x-4.3x faster end-to-end inference. These results demonstrate that our new latent collaboration framework enhances system-level reasoning quality while offering substantial efficiency gains without any additional training. Code and data are fully open-sourced at https://github.com/Gen-Verse/LatentMAS.