用于快速图像与视频生成的并行解码蒸馏

摘要

视频扩散模型或流模型的生成过程因迭代采样速度缓慢而计算成本高昂。当前最先进的加速方法严重依赖变分分数蒸馏（VSD）和对抗性损失，将扩散模型蒸馏为少步生成器。尽管这些训练方法能实现高质量视频生成，但其优化难度极大且易出现模式崩塌，导致视频多样性损失和运动缺乏。本文提出并行解码蒸馏（PDD）——一种简化且可扩展的轨迹蒸馏方法，用于扩散模型和流匹配模型的快速推理。我们的架构和训练流程与任何预训练模型兼容，并支持不同函数评估次数（NFE）的采样。PDD通过每轮网络评估预测多个去噪步骤来加速生成。从概念上看，它学习平均速度的表征，而无需通过雅可比向量积或有限差分近似回归其导数。我们的方法在LTX-2.3文生视频/音频、Wan 14B文生视频和Qwen-Image文生图任务上，以4-8次函数评估实现了最先进性能。此外，PDD在生成的视频多样性方面表现出显著提升。

English

Generation in video diffusion or flow models is computationally expensive due to the slow and iterative sampling process. Current state-of-the-art (SOTA) acceleration methods heavily rely on variational score distillation (VSD) and adversarial losses to distill diffusion models into few-step generators. Albeit achieving high-quality video generation, these training losses are notoriously hard to optimize and suffer from mode collapse, leading to loss of video diversity and lack of motion. In this paper, we introduce Parallel Decoding Distillation (PDD), a simplified and scalable trajectory-based distillation method for fast inference of diffusion and flow matching models. Our architecture and training procedure are compatible with any pre-trained model and support sampling with a varying number of function evaluations (NFE). PDD accelerates generation by predicting multiple denoising steps per network evaluation. Conceptually, it learns a representation of the mean velocity without regressing its derivative using JVPs or finite-difference approximations. Our method achieves SOTA performance with 4-8 NFE on LTX-2.3 Text-to-Video/Audio, Wan 14B Text-to-Video, and Qwen-Image Text-to-Image. Moreover, PDD presents a significant improvement in generated video diversity.