DynaVid：利用合成运动数据学习生成高动态视频

DynaVid: Learning to Generate Highly Dynamic Videos using Synthetic Motion Data

April 2, 2026

作者: Wonjoon Jin, Jiyun Won, Janghyeok Han, Qi Dai, Chong Luo, Seung-Hwan Baek, Sunghyun Cho

cs.AI

摘要

尽管近期取得进展，视频扩散模型在合成涉及剧烈动态运动或需要精细运动可控性的真实视频方面仍存在困难。核心局限在于常用训练数据集中此类样本的稀缺性。为此，我们推出DynaVid——一种利用合成运动数据进行训练的视频合成框架，该数据以光流形式呈现并通过计算机图形学管线渲染。该方法具有两大优势：首先，合成运动能提供真实数据难以获取的多样化运动模式和精确控制信号；其次，与具有人工外观的渲染视频不同，渲染光流仅编码运动信息且与外观解耦，从而避免模型重现合成视频的非自然观感。基于此理念，DynaVid采用双阶段生成框架：运动生成器先合成运动序列，再由运动引导的视频生成器根据运动条件生成视频帧。这种解耦架构使模型能够从合成数据学习动态运动模式，同时保持真实世界视频的视觉逼真度。我们在现有数据集特别受限的两个挑战性场景（剧烈人体运动生成与极端摄像机运动控制）上验证了框架有效性。大量实验表明，DynaVid显著提升了动态运动生成与摄像机运动控制的真实感与可控性。

English

Despite recent progress, video diffusion models still struggle to synthesize realistic videos involving highly dynamic motions or requiring fine-grained motion controllability. A central limitation lies in the scarcity of such examples in commonly used training datasets. To address this, we introduce DynaVid, a video synthesis framework that leverages synthetic motion data in training, which is represented as optical flow and rendered using computer graphics pipelines. This approach offers two key advantages. First, synthetic motion offers diverse motion patterns and precise control signals that are difficult to obtain from real data. Second, unlike rendered videos with artificial appearances, rendered optical flow encodes only motion and is decoupled from appearance, thereby preventing models from reproducing the unnatural look of synthetic videos. Building on this idea, DynaVid adopts a two-stage generation framework: a motion generator first synthesizes motion, and then a motion-guided video generator produces video frames conditioned on that motion. This decoupled formulation enables the model to learn dynamic motion patterns from synthetic data while preserving visual realism from real-world videos. We validate our framework on two challenging scenarios, vigorous human motion generation and extreme camera motion control, where existing datasets are particularly limited. Extensive experiments demonstrate that DynaVid improves the realism and controllability in dynamic motion generation and camera motion control.