透過噪聲感知引導緩解去噪生成模型中的噪聲偏移

摘要

現有的去噪生成模型依賴於求解離散化的反向時間隨機微分方程（SDEs）或常微分方程（ODEs）。本文中，我們揭示了這類模型中長期被忽視卻普遍存在的問題：預定義的噪聲水平與採樣過程中間狀態所編碼的實際噪聲水平之間的不匹配。我們將這種不匹配現象稱為噪聲偏移。通過實證分析，我們證明了噪聲偏移在現代擴散模型中廣泛存在，並呈現出系統性偏差，導致由於分佈外泛化和不準確的去噪更新而產生的次優生成結果。為解決這一問題，我們提出了噪聲感知指導（Noise Awareness Guidance, NAG），這是一種簡單而有效的校正方法，明確引導採樣軌跡與預定義的噪聲調度保持一致。我們進一步引入了NAG的無分類器變體，該變體通過噪聲條件丟棄聯合訓練一個噪聲條件模型和一個無噪聲條件模型，從而消除了對外部分類器的需求。包括ImageNet生成和多種監督微調任務在內的廣泛實驗表明，NAG持續緩解了噪聲偏移，並顯著提升了主流擴散模型的生成質量。

English

Existing denoising generative models rely on solving discretized reverse-time SDEs or ODEs. In this paper, we identify a long-overlooked yet pervasive issue in this family of models: a misalignment between the pre-defined noise level and the actual noise level encoded in intermediate states during sampling. We refer to this misalignment as noise shift. Through empirical analysis, we demonstrate that noise shift is widespread in modern diffusion models and exhibits a systematic bias, leading to sub-optimal generation due to both out-of-distribution generalization and inaccurate denoising updates. To address this problem, we propose Noise Awareness Guidance (NAG), a simple yet effective correction method that explicitly steers sampling trajectories to remain consistent with the pre-defined noise schedule. We further introduce a classifier-free variant of NAG, which jointly trains a noise-conditional and a noise-unconditional model via noise-condition dropout, thereby eliminating the need for external classifiers. Extensive experiments, including ImageNet generation and various supervised fine-tuning tasks, show that NAG consistently mitigates noise shift and substantially improves the generation quality of mainstream diffusion models.

透過噪聲感知引導緩解去噪生成模型中的噪聲偏移

Mitigating the Noise Shift for Denoising Generative Models via Noise Awareness Guidance

摘要

Support