可程式化世界模型

摘要

近期影片世界模型能產生日益逼真且具互動性的視覺體驗，然而缺乏可靠機制以在長時間互動中維持持久的世界狀態並強制執行可程式化規則。我們提出可程式化世界模型（Programmable World Model），一個將世界狀態演化與視覺觀測生成解耦的框架。代理程式會將自然語言指令轉譯為可執行程式，用以指定實體狀態與狀態轉移規則，從而能直接控制個別實體及其互動。輕量級引擎執行這些程式，以更新並維護明確且持久之全域世界狀態，包括畫面外實體與非視覺屬性。為了連結世界狀態與視覺生成，我們引入狀態增強 3D 有向邊界框（OBBs）作為中間表示。此表示連同目標攝影機軌跡，會被確定性地編譯為像素對齊的時空條件訊號，供作為生成式渲染器的預訓練影片模型使用。此設計讓使用者能建立具有預定義機制的可玩遊戲、直接控制個別實體，並在整個遊戲過程中維持持久的世界狀態。我們進一步提出 CombatStateBench，一個用於評估可程式化世界模型的基準。在 CombatStateBench 上，我們的方法達到 94% 數量準確率與 98% 狀態準確率，大幅超越現有的互動式影片世界模型，同時支援連貫的長時程生成。這些結果證明，將明確狀態演化與生成式渲染分離，對於建構持久且可程式化的世界是有效的。

English

Recent video world models generate increasingly realistic and interactive visual experiences, yet lack reliable mechanisms for maintaining persistent world state and enforcing programmable rules over extended interactions. We introduce Programmable World Model, a framework that decouples world-state evolution from visual observation generation. An agent translates natural-language instructions into executable programs that specify entity states and state-transition rules, enabling direct control over individual entities and their interactions. A lightweight engine executes these programs to update and maintain an explicit, persistent global world state, including off-screen entities and non-visual attributes. To connect world state with visual generation, we introduce state-augmented 3D oriented bounding boxes (OBBs) as an intermediate representation. This representation, together with the target camera trajectory, is deterministically compiled into pixel-aligned spatiotemporal conditioning signals for a pretrained video model serving as the generative renderer. This design allows users to create playable games with predefined mechanics, direct control over individual entities, and persistent world state throughout gameplay. We further introduce CombatStateBench, a benchmark for evaluating programmable world models. On CombatStateBench, our method achieves 94% Count Accuracy and 98% State Accuracy, substantially outperforming existing interactive video world models while supporting coherent long-horizon generation. These results demonstrate the effectiveness of separating explicit state evolution from generative rendering for building persistent, programmable worlds.