多智能体协作中的思维通信

摘要

自然语言虽长期支撑着人类协作，但其存在信息损耗、歧义性与间接性等局限，制约了集体智能的潜力。尽管机器不受此类限制，当前大多数基于大语言模型的多智能体系统仍仅依赖自然语言交互，通过词元或其嵌入向量进行通信。为突破语言桎梏，我们提出"思维通信"新范式，使智能体能够实现类似心灵感应的直接意识交互。为系统化揭示这些潜在思维，我们将其形式化为广义潜变量模型：智能体状态由底层思维的未知函数生成。我们证明，在无辅助信息的非参数设定下，任意智能体对之间的共享与私有潜在思维均可被识别；且思维共享的全局结构——包括哪些智能体共享何种思维及其关联模式——亦可基于理论保证被还原。基于该理论框架，我们开发出在通信前从所有智能体提取潜在思维，并为每个智能体分配相关思维及其共享模式的系统。此范式可自然扩展至大语言模型之外的全模态领域，因为多数观测数据均源自隐藏的生成过程。在合成与真实场景下的实验验证了理论正确性，并展现出思维通信的协同优势。本研究旨在揭示挖掘隐藏世界的潜力——毕竟仅靠表层观测，即便拥有海量算力与数据，诸多挑战仍将无解。

English

Natural language has long enabled human cooperation, but its lossy, ambiguous, and indirect nature limits the potential of collective intelligence. While machines are not subject to these constraints, most LLM-based multi-agent systems still rely solely on natural language, exchanging tokens or their embeddings. To go beyond language, we introduce a new paradigm, thought communication, which enables agents to interact directly mind-to-mind, akin to telepathy. To uncover these latent thoughts in a principled way, we formalize the process as a general latent variable model, where agent states are generated by an unknown function of underlying thoughts. We prove that, in a nonparametric setting without auxiliary information, both shared and private latent thoughts between any pair of agents can be identified. Moreover, the global structure of thought sharing, including which agents share which thoughts and how these relationships are structured, can also be recovered with theoretical guarantees. Guided by the established theory, we develop a framework that extracts latent thoughts from all agents prior to communication and assigns each agent the relevant thoughts, along with their sharing patterns. This paradigm naturally extends beyond LLMs to all modalities, as most observational data arise from hidden generative processes. Experiments on both synthetic and real-world benchmarks validate the theory and demonstrate the collaborative advantages of thought communication. We hope this work illuminates the potential of leveraging the hidden world, as many challenges remain unsolvable through surface-level observation alone, regardless of compute or data scale.

多智能体协作中的思维通信

Thought Communication in Multiagent Collaboration

摘要

Support