尺度感知的扩散模型蒸馏

Scale-wise Distillation of Diffusion Models

March 20, 2025

作者: Nikita Starodubcev, Denis Kuznedelev, Artem Babenko, Dmitry Baranchuk

cs.AI

摘要

我们提出了SwD，一种面向扩散模型（DMs）的尺度间蒸馏框架，该框架有效利用了下一尺度预测的思想，专为基于扩散的少步生成器设计。具体而言，SwD受到近期将扩散过程与隐式频谱自回归相关联的见解启发。我们假设DMs可以在较低的数据分辨率下启动生成过程，并在每一步去噪过程中逐步上采样样本，而不会牺牲性能，同时显著降低计算成本。SwD巧妙地将这一理念融入现有的基于分布匹配的扩散蒸馏方法中。此外，我们通过引入一种新颖的补丁损失，丰富了分布匹配方法家族，该损失强制实现与目标分布更细粒度的相似性。当应用于最先进的文本到图像扩散模型时，SwD在接近两次全分辨率步骤的推理时间内，显著超越了同等计算预算下的对比方法，这一优势通过自动化指标和人类偏好研究得到了验证。

English

We present SwD, a scale-wise distillation framework for diffusion models (DMs), which effectively employs next-scale prediction ideas for diffusion-based few-step generators. In more detail, SwD is inspired by the recent insights relating diffusion processes to the implicit spectral autoregression. We suppose that DMs can initiate generation at lower data resolutions and gradually upscale the samples at each denoising step without loss in performance while significantly reducing computational costs. SwD naturally integrates this idea into existing diffusion distillation methods based on distribution matching. Also, we enrich the family of distribution matching approaches by introducing a novel patch loss enforcing finer-grained similarity to the target distribution. When applied to state-of-the-art text-to-image diffusion models, SwD approaches the inference times of two full resolution steps and significantly outperforms the counterparts under the same computation budget, as evidenced by automated metrics and human preference studies.