多智能体协作中的思维通信

摘要

自然语言虽长期维系着人类协作，但其有损、模糊与间接的特性限制了集体智能的潜力。尽管机器不受此类限制，当前大多数基于大语言模型的多智能体系统仍仅依赖自然语言进行词元或其嵌入向量的交换。为突破语言局限，我们提出"思维通信"新范式，使智能体能够实现类似心灵感应的直接意识交互。为系统化揭示这些潜在思维，我们将其形式化为广义潜变量模型：智能体状态由底层思维的未知函数生成。我们证明在无辅助信息的非参数设定下，任意智能体对之间的共享与私有潜在思维皆可识别；且思维共享的全局结构（包括哪些智能体共享何种思维及其关联模式）亦可被理论保证地还原。基于该理论框架，我们开发出在通信前从所有智能体提取潜在思维，并为每个智能体分配相关思维及其共享模式的系统。此范式自然延伸至大语言模型之外的所有模态，因多数观测数据皆源自隐藏的生成过程。合成与真实场景的基准实验验证了理论，并证明思维通信的协作优势。本研究旨在揭示挖掘隐藏世界的潜力——诸多挑战仅凭表层观测终难解决，无论算力或数据规模如何扩展。

English

Natural language has long enabled human cooperation, but its lossy, ambiguous, and indirect nature limits the potential of collective intelligence. While machines are not subject to these constraints, most LLM-based multi-agent systems still rely solely on natural language, exchanging tokens or their embeddings. To go beyond language, we introduce a new paradigm, thought communication, which enables agents to interact directly mind-to-mind, akin to telepathy. To uncover these latent thoughts in a principled way, we formalize the process as a general latent variable model, where agent states are generated by an unknown function of underlying thoughts. We prove that, in a nonparametric setting without auxiliary information, both shared and private latent thoughts between any pair of agents can be identified. Moreover, the global structure of thought sharing, including which agents share which thoughts and how these relationships are structured, can also be recovered with theoretical guarantees. Guided by the established theory, we develop a framework that extracts latent thoughts from all agents prior to communication and assigns each agent the relevant thoughts, along with their sharing patterns. This paradigm naturally extends beyond LLMs to all modalities, as most observational data arise from hidden generative processes. Experiments on both synthetic and real-world benchmarks validate the theory and demonstrate the collaborative advantages of thought communication. We hope this work illuminates the potential of leveraging the hidden world, as many challenges remain unsolvable through surface-level observation alone, regardless of compute or data scale.

多智能体协作中的思维通信

Thought Communication in Multiagent Collaboration

摘要

Support