快速视频生成的过渡匹配蒸馏法

摘要

大规模视频扩散与流模型已在高质量视频生成领域取得显著成功，但由于其多步采样过程效率低下，在实时交互应用中的使用仍受限。本研究提出过渡匹配蒸馏（TMD）框架，通过将视频扩散模型蒸馏为高效少步生成器来解决该问题。TMD的核心思想是将扩散模型的多步去噪轨迹与少步概率转移过程相匹配，其中每个转移步骤通过轻量级条件流模型实现。为实现高效蒸馏，我们将原始扩散主干网络分解为两部分：（1）主骨干网络（包含多数早期层），用于在外部转移步骤中提取语义表征；（2）流头部网络（由最后几层构成），利用这些表征执行多重内部流更新。给定预训练视频扩散模型，我们首先为其引入流头部结构，将其适配为条件流映射。随后通过在每个转移步骤中实施流头部展开策略，对带有流头部的学生模型进行分布匹配蒸馏。基于Wan2.1的1.3B和140亿参数文本到视频模型的广泛实验表明，TMD在生成速度与视觉质量间实现了灵活而优越的平衡。特别值得注意的是，在相近推理成本下，TMD在视觉保真度和提示词遵循度方面均优于现有蒸馏模型。项目页面：https://research.nvidia.com/labs/genair/tmd

English

Large video diffusion and flow models have achieved remarkable success in high-quality video generation, but their use in real-time interactive applications remains limited due to their inefficient multi-step sampling process. In this work, we present Transition Matching Distillation (TMD), a novel framework for distilling video diffusion models into efficient few-step generators. The central idea of TMD is to match the multi-step denoising trajectory of a diffusion model with a few-step probability transition process, where each transition is modeled as a lightweight conditional flow. To enable efficient distillation, we decompose the original diffusion backbone into two components: (1) a main backbone, comprising the majority of early layers, that extracts semantic representations at each outer transition step; and (2) a flow head, consisting of the last few layers, that leverages these representations to perform multiple inner flow updates. Given a pretrained video diffusion model, we first introduce a flow head to the model, and adapt it into a conditional flow map. We then apply distribution matching distillation to the student model with flow head rollout in each transition step. Extensive experiments on distilling Wan2.1 1.3B and 14B text-to-video models demonstrate that TMD provides a flexible and strong trade-off between generation speed and visual quality. In particular, TMD outperforms existing distilled models under comparable inference costs in terms of visual fidelity and prompt adherence. Project page: https://research.nvidia.com/labs/genair/tmd

快速视频生成的过渡匹配蒸馏法

Transition Matching Distillation for Fast Video Generation

摘要

Support