Glance：单样本加速扩散模型

摘要

擴散模型在圖像生成領域取得了顯著成功，但其部署仍受制於高昂的計算成本和繁瑣的推理步驟。先前關於少步數蒸餾的研究試圖通過訓練緊湊的學生模型來跳過冗餘步驟，但往往面臨沉重的再訓練成本與泛化能力下降的問題。本研究提出全新視角：採用智能非均勻加速策略，對早期語義階段施加較小加速比，而對後期冗餘階段實施更大加速比。我們通過配備專注於慢速與快速去噪階段的雙專家模型來實現這一階段感知策略。令人驚訝的是，無需投入大量資源重新訓練學生模型，僅需為基礎模型配備輕量級LoRA適配器即可同時實現高效加速與強泛化能力。我們將這兩種適配器命名為Slow-LoRA與Fast-LoRA。大量實驗表明，該方法在保持多樣化基準測試中可視質量的同時，可實現相較基礎模型最高5倍的加速效果。值得注意的是，LoRA專家模型僅需使用1%的樣本在單張V100顯卡上訓練一小時，所得模型對未見過的提示詞仍展現出強大的泛化能力。

English

Diffusion models have achieved remarkable success in image generation, yet their deployment remains constrained by the heavy computational cost and the need for numerous inference steps. Previous efforts on fewer-step distillation attempt to skip redundant steps by training compact student models, yet they often suffer from heavy retraining costs and degraded generalization. In this work, we take a different perspective: we accelerate smartly, not evenly, applying smaller speedups to early semantic stages and larger ones to later redundant phases. We instantiate this phase-aware strategy with two experts that specialize in slow and fast denoising phases. Surprisingly, instead of investing massive effort in retraining student models, we find that simply equipping the base model with lightweight LoRA adapters achieves both efficient acceleration and strong generalization. We refer to these two adapters as Slow-LoRA and Fast-LoRA. Through extensive experiments, our method achieves up to 5 acceleration over the base model while maintaining comparable visual quality across diverse benchmarks. Remarkably, the LoRA experts are trained with only 1 samples on a single V100 within one hour, yet the resulting models generalize strongly on unseen prompts.

Glance：单样本加速扩散模型

Glance: Accelerating Diffusion Models with 1 Sample

摘要

Support