多智能体系统中的潜在协作

Latent Collaboration in Multi-Agent Systems

November 25, 2025

作者: Jiaru Zou, Xiyuan Yang, Ruizhong Qiu, Gaotang Li, Katherine Tieu, Pan Lu, Ke Shen, Hanghang Tong, Yejin Choi, Jingrui He, James Zou, Mengdi Wang, Ling Yang

cs.AI

摘要

多智能体系统（MAS）将大语言模型（LLM）从独立的单模型推理扩展至协同的系统级智能。现有LLM智能体依赖基于文本的中介进行推理与通信，而我们通过使模型能在连续潜空间内直接协作更进一步。本文提出LatentMAS——一种支持LLM智能体间纯潜空间协作的端到端免训练框架。在LatentMAS中，每个智能体首先通过末层隐藏嵌入进行自回归潜思维生成，共享的潜工作记忆则保存并传递各智能体的内部表征，确保无损信息交换。理论分析表明，相较于传统基于文本的MAS，LatentMAS能以显著更低的复杂度实现更高表达力与无损信息保存。此外，在涵盖数学科学推理、常识理解与代码生成的9个综合基准测试中，LatentMAS持续优于强单模型及文本MAS基线，准确率最高提升14.6%，输出令牌使用量减少70.8%-83.7%，端到端推理速度提升4-4.3倍。这些结果证明，我们的新潜协作框架在提升系统级推理质量的同时，无需额外训练即可实现显著效率增益。代码与数据已开源：https://github.com/Gen-Verse/LatentMAS。

English

Multi-agent systems (MAS) extend large language models (LLMs) from independent single-model reasoning to coordinative system-level intelligence. While existing LLM agents depend on text-based mediation for reasoning and communication, we take a step forward by enabling models to collaborate directly within the continuous latent space. We introduce LatentMAS, an end-to-end training-free framework that enables pure latent collaboration among LLM agents. In LatentMAS, each agent first performs auto-regressive latent thoughts generation through last-layer hidden embeddings. A shared latent working memory then preserves and transfers each agent's internal representations, ensuring lossless information exchange. We provide theoretical analyses establishing that LatentMAS attains higher expressiveness and lossless information preservation with substantially lower complexity than vanilla text-based MAS. In addition, empirical evaluations across 9 comprehensive benchmarks spanning math and science reasoning, commonsense understanding, and code generation show that LatentMAS consistently outperforms strong single-model and text-based MAS baselines, achieving up to 14.6% higher accuracy, reducing output token usage by 70.8%-83.7%, and providing 4x-4.3x faster end-to-end inference. These results demonstrate that our new latent collaboration framework enhances system-level reasoning quality while offering substantial efficiency gains without any additional training. Code and data are fully open-sourced at https://github.com/Gen-Verse/LatentMAS.