遊戲工廠：使用生成互動影片創建新遊戲

GameFactory: Creating New Games with Generative Interactive Videos

January 14, 2025

作者: Jiwen Yu, Yiran Qin, Xintao Wang, Pengfei Wan, Di Zhang, Xihui Liu

cs.AI

摘要

生成式遊戲引擎具有革命性的潛力，可以自主創建新內容並減少手動工作量。然而，現有基於視頻的遊戲生成方法未能解決場景泛化的關鍵挑戰，限制了其應用範圍，僅適用於具有固定風格和場景的現有遊戲。本文提出了GameFactory，一個專注於探索遊戲視頻生成中場景泛化的框架。為了創建全新和多樣化的遊戲，我們利用在開放域視頻數據上訓練的預訓練視頻擴散模型。為了彌合開放域先驗和小規模遊戲數據集之間的領域差距，我們提出了一種多階段訓練策略，將遊戲風格學習與動作控制分離，保持開放域泛化同時實現動作可控性。利用Minecraft作為我們的數據來源，我們釋出了GF-Minecraft，一個高質量且多樣化的動作標註視頻數據集供研究使用。此外，我們擴展了我們的框架，實現自回歸動作可控的遊戲視頻生成，實現無限長度的互動遊戲視頻製作。實驗結果表明，GameFactory有效生成開放域、多樣化且動作可控的遊戲視頻，代表了AI驅動的遊戲生成的重要進步。我們的數據集和項目頁面可在https://vvictoryuki.github.io/gamefactory/公開獲取。

English

Generative game engines have the potential to revolutionize game development by autonomously creating new content and reducing manual workload. However, existing video-based game generation methods fail to address the critical challenge of scene generalization, limiting their applicability to existing games with fixed styles and scenes. In this paper, we present GameFactory, a framework focused on exploring scene generalization in game video generation. To enable the creation of entirely new and diverse games, we leverage pre-trained video diffusion models trained on open-domain video data. To bridge the domain gap between open-domain priors and small-scale game dataset, we propose a multi-phase training strategy that decouples game style learning from action control, preserving open-domain generalization while achieving action controllability. Using Minecraft as our data source, we release GF-Minecraft, a high-quality and diversity action-annotated video dataset for research. Furthermore, we extend our framework to enable autoregressive action-controllable game video generation, allowing the production of unlimited-length interactive game videos. Experimental results demonstrate that GameFactory effectively generates open-domain, diverse, and action-controllable game videos, representing a significant step forward in AI-driven game generation. Our dataset and project page are publicly available at https://vvictoryuki.github.io/gamefactory/.