从像素到状态：重新思考交互世界模型作为游戏引擎

摘要

构建能够根据玩家动作连贯响应的交互世界，长期以来一直是计算机图形学、游戏和人工智能领域的共同目标。近期视频生成模型提供了一种数据驱动的方法，通过根据用户动作预测未来观测结果来实现这一目标，并越来越被视为潜在的下一代游戏引擎。然而，实现真正交互的游戏世界需要遵循演化游戏条件规则所产生的交互结果、能够在长时间跨度内持续存在的后果，以及实时运行的生成循环。传统游戏引擎通过循环的"动作-状态-观察"流程实现这些特性：玩家动作根据预定义规则更新显式游戏状态，并从更新后的状态渲染出观察结果。本文以此循环为组织框架，从四个维度考察交互式游戏世界建模：玩家动作控制、游戏状态动态性、状态-观察持续性，以及实时交互生成。针对每个维度，我们从交互游戏世界所需的能力出发，将现有方法归入代表性类别，并讨论各类别的优势与权衡。作为分析补充，我们为《黑神话：悟空》设计了可扩展数据引擎，收集了超过90小时游戏视频，其中包含帧对齐的玩家动作、真实游戏状态及视觉观察，并配有结构化与语义化注释，为状态感知的游戏世界建模提供资源。希望本文能够清晰呈现该领域现状，推动交互式游戏世界的发展。

English

Building interactive worlds that respond coherently to player actions has long been a shared goal of computer graphics, games, and artificial intelligence. Recent video generative models provide a data-driven route toward this goal by predicting future observations conditioned on user actions, and are increasingly regarded as potential next-generation game engines. Realizing a genuinely interactive game world, however, requires interaction outcomes that follow rules over evolving game conditions, consequences that persist over long horizons, and a generation loop that operates in real time. Conventional game engines realize these properties through a recurrent action-state-observation loop, in which player actions update an explicit game state according to predefined rules and observations are rendered from the resulting state. Taking this loop as an organizing lens, this paper examines interactive game world modeling along four dimensions: player action control, game state dynamics, state-observation persistence, and real-time interactive generation. For each dimension, we start from the capabilities required by an interactive game world, group existing approaches into representative families, and discuss the strengths and trade-offs of each family. Complementing this analysis, we present a scalable data engine for Black Myth: Wukong that collects over 90 hours of gameplay with frame-aligned player actions, ground-truth game states, and visual observations, together with structured and semantic annotations, as a resource for state-aware game world modeling. We hope this paper offers a clear picture of where the field stands and fosters progress toward interactive game worlds.