交互式生成视频技术综述

A Survey of Interactive Generative Video

April 30, 2025

作者: Jiwen Yu, Yiran Qin, Haoxuan Che, Quande Liu, Xintao Wang, Pengfei Wan, Di Zhang, Kun Gai, Hao Chen, Xihui Liu

cs.AI

摘要

交互式生成视频（IGV）作为一项关键技术应运而生，以满足各领域对高质量、互动性视频内容日益增长的需求。本文中，我们将IGV定义为一种结合生成能力以产出多样化高质量视频内容，并具备互动功能的技术，这些功能通过控制信号和响应反馈实现用户参与。我们调研了当前IGV的应用现状，聚焦于三大领域：1）游戏领域，IGV支持在虚拟世界中的无限探索；2）具身人工智能领域，IGV作为物理感知的环境合成器，用于训练智能体在多模态交互中应对动态变化的场景；3）自动驾驶领域，IGV提供闭环模拟能力，用于安全关键测试与验证。为指引未来发展，我们提出一个综合框架，将理想的IGV系统分解为五个核心模块：生成、控制、记忆、动态与智能。此外，我们系统分析了实现理想IGV系统各组件所面临的技术挑战与未来方向，如实现实时生成、支持开放域控制、保持长期一致性、模拟精确物理以及整合因果推理。我们相信，这一系统分析将促进IGV领域的未来研究与开发，最终推动该技术向更复杂、更实用的应用迈进。

English

Interactive Generative Video (IGV) has emerged as a crucial technology in response to the growing demand for high-quality, interactive video content across various domains. In this paper, we define IGV as a technology that combines generative capabilities to produce diverse high-quality video content with interactive features that enable user engagement through control signals and responsive feedback. We survey the current landscape of IGV applications, focusing on three major domains: 1) gaming, where IGV enables infinite exploration in virtual worlds; 2) embodied AI, where IGV serves as a physics-aware environment synthesizer for training agents in multimodal interaction with dynamically evolving scenes; and 3) autonomous driving, where IGV provides closed-loop simulation capabilities for safety-critical testing and validation. To guide future development, we propose a comprehensive framework that decomposes an ideal IGV system into five essential modules: Generation, Control, Memory, Dynamics, and Intelligence. Furthermore, we systematically analyze the technical challenges and future directions in realizing each component for an ideal IGV system, such as achieving real-time generation, enabling open-domain control, maintaining long-term coherence, simulating accurate physics, and integrating causal reasoning. We believe that this systematic analysis will facilitate future research and development in the field of IGV, ultimately advancing the technology toward more sophisticated and practical applications.