從像素到狀態：重新思考互動世界模型作為遊戲引擎

摘要

長期以來，建立能根據玩家行動連貫回應的互動世界，始終是電腦圖形學、遊戲與人工智慧的共同目標。近期影片生成模型透過預測基於使用者行動的未來觀察結果，為此目標提供了數據驅動途徑，並日益被視為潛在的次世代遊戲引擎。然而，要實現真正可互動的遊戲世界，需要互動結果能遵循遊戲動態狀態的規則、後果能長時段持續存在，且生成循環能即時運作。傳統遊戲引擎透過循環的「動作-狀態-觀察」機制實現這些特性：玩家動作按照預設規則更新明確的遊戲狀態，再從更新後的狀態渲染出觀察畫面。本文以該循環為組織框架，從四個面向探討互動遊戲世界建模：玩家動作控制、遊戲狀態動態、狀態-觀察持續性，以及即時互動生成。針對每個面向，我們先界定互動遊戲世界所需的關鍵能力，再將現有方法歸納為具代表性的類別，並討論各類別的優勢與取捨。為補充此分析，我們針對《黑神話：悟空》開發了一套可擴展的數據引擎，收集超過90小時的遊戲過程，包含逐幀對齊的玩家動作、真實遊戲狀態與視覺觀察記錄，並附有結構化及語義標註，作為狀態感知遊戲世界建模的資源。期望本文能清晰描繪此領域的現狀，並推動互動遊戲世界的相關進展。

English

Building interactive worlds that respond coherently to player actions has long been a shared goal of computer graphics, games, and artificial intelligence. Recent video generative models provide a data-driven route toward this goal by predicting future observations conditioned on user actions, and are increasingly regarded as potential next-generation game engines. Realizing a genuinely interactive game world, however, requires interaction outcomes that follow rules over evolving game conditions, consequences that persist over long horizons, and a generation loop that operates in real time. Conventional game engines realize these properties through a recurrent action-state-observation loop, in which player actions update an explicit game state according to predefined rules and observations are rendered from the resulting state. Taking this loop as an organizing lens, this paper examines interactive game world modeling along four dimensions: player action control, game state dynamics, state-observation persistence, and real-time interactive generation. For each dimension, we start from the capabilities required by an interactive game world, group existing approaches into representative families, and discuss the strengths and trade-offs of each family. Complementing this analysis, we present a scalable data engine for Black Myth: Wukong that collects over 90 hours of gameplay with frame-aligned player actions, ground-truth game states, and visual observations, together with structured and semantic annotations, as a resource for state-aware game world modeling. We hope this paper offers a clear picture of where the field stands and fosters progress toward interactive game worlds.