带有世界状态寄存器的流式多智能体自回归扩散模型

摘要

多智能体交互世界模型不仅需要生成一致的观测结果，还必须维护跨智能体持久存在且随视角演化的世界状态。现有的自回归视频扩散流水线将观测历史作为条件上下文传递，这使得在多智能体与多视角场景中难以维持共享状态。我们提出WorldWeaver (W²)，一种流式多智能体视频扩散模型，其通过引入跨智能体世界状态寄存器来增强生成过程：这些可学习令牌存储共享世界信息，追踪个体智能体状态，并在每个生成片段后动态更新。我们为这些寄存器提供涵盖个体智能体状态、全局视角（包括鸟瞰图）及场景文本的监督信号。此外，我们采用混合Transformer架构对模型进行改进，该架构使用独立的权重分别处理世界状态建模与视觉帧建模。在双智能体Minecraft视频生成任务中的大量实验表明，显式的世界状态建模提升了逻辑一致性与生成质量。

English

Multi-agent interactive world models should not only generate consistent observations, but also maintain world states that persist across agents and evolve across views. Existing autoregressive video diffusion pipelines carry forward observation history as conditioning context, which makes shared state difficult to maintain in multi-agent and multi-view settings. We present WorldWeaver (W^2), a streaming multi-agent video diffusion model that augments rollout with cross-agent world state registers: learnable tokens that store shared world information, track individual agent status, and are dynamically updated after each generated chunk. We ground these registers with supervision signals spanning individual agent status, global state views including bird's-eye views, and scene text. We further improve the architecture with a Mixture-of-Transformers design that uses separate weights for world state modeling and visual frame modeling. Extensive experiments in two-agent Minecraft video generation show that explicit world-state modeling improves logical consistency and generation quality.