Game2World引擎：解锁野外游戏视频以用于世界模型训练

摘要

视频游戏为视频世界模型提供了可扩展的训练数据来源，包含多样化的环境、复杂的交互以及丰富的野外游戏视频。然而，原始游戏录像将游戏世界与屏幕空间界面纠缠在一起，引入了游戏特定偏差和无关动态，从而阻碍了世界模型的训练。为解决这一问题，我们提出了GameUI-Taxonomy和G2WEngine，这是一个全栈框架，规范了游戏UI的定位与去除。G2WEngine能够自动从真实游戏视频中提取可复用的UI资产，并在干净视频素材上合成时间上连贯的UI覆盖层。利用该引擎，我们构建了Game2World数据集，包含96K个具有精确重建目标的合成配对视频，以及来自303款游戏的1,079个野外片段，用于真实场景评估。其资产库包含来自1,010个代表性游戏画面的5,132个经过验证的UI元素，涵盖21个分类类别。基于Game2World，我们提出了GameCleaner，一种无掩膜的游戏UI去除模型，它结合了多模态语义理解与视频编辑能力。与基于掩膜的方法不同，GameCleaner直接识别并移除各种HUD元素，同时保留底层场景内容和时间动态。在受控试点实验中，基于无UI游戏画面训练的世界模型相比基于UI叠加数据训练的模型，在整体VideoReward上提升了6.83%。在UI去除评估中，GameCleaner在合成视频上实现了95.36的平均AAR，比最强的时间掩膜基线高出57.3%，并在野外数据上获得了最佳AAR 80.05，同时实现了99.8%的背景保留率。这些结果展示了将互联网游戏视频转化为高质量世界模型训练数据的可扩展潜力。代码、数据集和模型将在 https://github.com/Dongping-Chen/Game2World 上提供。

English

Video games provide a scalable source of training data for video world models, offering diverse environments, complex interactions, and abundant in-the-wild gameplay videos. However, raw gameplay footage entangles the game world with screen-space interfaces, introducing game-specific biases and irrelevant dynamics that hinder world-model training. To address this problem, we introduce GameUI-Taxonomy and G2WEngine, a full-stack framework that formalizes gameplay UI grounding and removal. G2WEngine automatically extracts reusable UI assets from real gameplay videos and synthesizes temporally coherent UI overlays on clean footage. Using this engine, we construct Game2World, comprising 96K synthetic paired videos with precise reconstruction targets and 1,079 in-the-wild clips from 303 games for realistic evaluation. Its asset library contains 5,132 verified UI elements across 21 taxonomy categories, collected from 1,010 representative gameplay frames. Based on Game2World, we propose GameCleaner, a mask-free gameplay UI removal model that combines multimodal semantic understanding with video editing capabilities. Unlike mask-based methods, GameCleaner directly identifies and removes diverse HUD elements while preserving the underlying scene content and temporal dynamics. In a controlled pilot, world models trained on UI-free gameplay improve overall VideoReward by 6.83% over those trained on UI-overlaid data. On UI-removal evaluation, GameCleaner achieves an average AAR of 95.36 on synthetic videos, outperforming the strongest temporal mask baseline by 57.3%, and obtains the best in-the-wild AAR of 80.05 with 99.8 background preservation. These results demonstrate the scalable potential of transforming Internet gameplay videos into high-quality world-model training data. Code, dataset, and model will be available at https://github.com/Dongping-Chen/Game2World.