並行解碼蒸餾用於快速圖像與視頻生成

摘要

影片擴散或流模型中的生成因取樣過程緩慢且迭代而計算成本高昂。當前最先進（SOTA）的加速方法高度依賴變分分數蒸餾（VSD）與對抗性損失，將擴散模型蒸餾為少步生成器。然而，這類訓練損失雖能生成高品質影片，卻因難以優化且易發生模式崩潰而聲名狼藉，導致影片多樣性喪失及動態不足。本文提出並行解碼蒸餾（PDD），一種簡化且可擴展的基於軌跡蒸餾方法，用於加速擴散與流匹配模型的推論。我們的架構與訓練流程可適用於任何預訓練模型，並支援不同函數評估次數（NFE）的取樣。PDD透過在每次網路評估中預測多個去噪步驟來加速生成。概念上，它學習平均速度的表示，而無需使用雅可比向量積（JVPs）或有限差分近似來回歸其導數。我們的方法在LTX-2.3文字轉影片/音訊、Wan 14B文字轉影片及Qwen-Image文字轉圖像上，以4-8 NFE達到SOTA性能。此外，PDD在生成影片的多樣性方面展現了顯著提升。

English

Generation in video diffusion or flow models is computationally expensive due to the slow and iterative sampling process. Current state-of-the-art (SOTA) acceleration methods heavily rely on variational score distillation (VSD) and adversarial losses to distill diffusion models into few-step generators. Albeit achieving high-quality video generation, these training losses are notoriously hard to optimize and suffer from mode collapse, leading to loss of video diversity and lack of motion. In this paper, we introduce Parallel Decoding Distillation (PDD), a simplified and scalable trajectory-based distillation method for fast inference of diffusion and flow matching models. Our architecture and training procedure are compatible with any pre-trained model and support sampling with a varying number of function evaluations (NFE). PDD accelerates generation by predicting multiple denoising steps per network evaluation. Conceptually, it learns a representation of the mean velocity without regressing its derivative using JVPs or finite-difference approximations. Our method achieves SOTA performance with 4-8 NFE on LTX-2.3 Text-to-Video/Audio, Wan 14B Text-to-Video, and Qwen-Image Text-to-Image. Moreover, PDD presents a significant improvement in generated video diversity.