Game2World引擎：解鎖在野遊戲影片以訓練世界模型

摘要

電子遊戲為視頻世界模型提供了可擴展的訓練數據來源，涵蓋多樣化的環境、複雜的交互以及豐富的真實遊戲視頻。然而，原始遊戲畫面將遊戲世界與屏幕空間界面糾纏在一起，引入遊戲特定偏差和無關動態，阻礙了世界模型的訓練。為解決這一問題，我們提出了GameUI-Taxonomy和G2WEngine——一個全棧框架，用於形式化遊戲UI的定位與移除。G2WEngine能自動從真實遊戲視頻中提取可重用UI資產，並在乾淨畫面上合成時間一致的UI疊加層。利用該引擎，我們構建了Game2World數據集，包含96K合成配對視頻（帶有精確重建目標）以及來自303款遊戲的1,079個真實場景片段，用於真實場景評估。其資產庫包含來自1,010個代表性遊戲幀的5,132個已驗證UI元素，涵蓋21個分類類別。基於Game2World，我們提出了GameCleaner——一種無遮罩的遊戲UI移除模型，結合多模態語義理解與視頻編輯能力。與基於遮罩的方法不同，GameCleaner直接識別並移除多樣化的HUD元素，同時保留底層場景內容和時間動態。在一項受控試點實驗中，使用無UI遊戲畫面訓練的世界模型，其整體VideoReward比使用帶UI疊加數據訓練的模型高出6.83%。在UI移除評估中，GameCleaner在合成視頻上達到了95.36的平均AAR，比最強的時序遮罩基線高出57.3%，並在真實場景中取得了最佳的80.05 AAR，背景保留率達99.8%。這些結果展示了將互聯網遊戲視頻轉化為高質量世界模型訓練數據的巨大潛力。代碼、數據集和模型將在https://github.com/Dongping-Chen/Game2World提供。

English

Video games provide a scalable source of training data for video world models, offering diverse environments, complex interactions, and abundant in-the-wild gameplay videos. However, raw gameplay footage entangles the game world with screen-space interfaces, introducing game-specific biases and irrelevant dynamics that hinder world-model training. To address this problem, we introduce GameUI-Taxonomy and G2WEngine, a full-stack framework that formalizes gameplay UI grounding and removal. G2WEngine automatically extracts reusable UI assets from real gameplay videos and synthesizes temporally coherent UI overlays on clean footage. Using this engine, we construct Game2World, comprising 96K synthetic paired videos with precise reconstruction targets and 1,079 in-the-wild clips from 303 games for realistic evaluation. Its asset library contains 5,132 verified UI elements across 21 taxonomy categories, collected from 1,010 representative gameplay frames. Based on Game2World, we propose GameCleaner, a mask-free gameplay UI removal model that combines multimodal semantic understanding with video editing capabilities. Unlike mask-based methods, GameCleaner directly identifies and removes diverse HUD elements while preserving the underlying scene content and temporal dynamics. In a controlled pilot, world models trained on UI-free gameplay improve overall VideoReward by 6.83% over those trained on UI-overlaid data. On UI-removal evaluation, GameCleaner achieves an average AAR of 95.36 on synthetic videos, outperforming the strongest temporal mask baseline by 57.3%, and obtains the best in-the-wild AAR of 80.05 with 99.8 background preservation. These results demonstrate the scalable potential of transforming Internet gameplay videos into high-quality world-model training data. Code, dataset, and model will be available at https://github.com/Dongping-Chen/Game2World.