減小擴散模型曝光偏差的頻譜先驗

摘要

擴散模型在迭代取樣過程中通常會累積誤差，此現象常被稱為暴露偏差。我們揭示了訓練與推理之間存在系統性的頻率相關偏差，可將其解讀為頻率相關的信噪比誤差。關鍵在於，此偏差的方向會隨模型與時間步階而變化，顯示固定修正規則無法通用。我們提出頻譜對齊法（Spectral Alignment, SPA），這是一種輕量級、基於引導的方法，可將中間預測的功率譜校準至預先計算的先驗。該方法包含兩個階段：(1) 根據訓練資料離線擬合參數化頻譜模型；(2) 在推理階段透過基於快速傅立葉轉換（FFT）的高效率梯度計算進行引導。SPA僅引入極小的計算開銷（約3-4%），並可與無分類器引導（Classifier-Free Guidance, CFG）互補。我們在多種架構上展現一致性的改進，涵蓋像素空間模型（DDPM、ADM）、潛變數擴散模型（SD2.0、SDXL）以及流匹配模型（SD3.5、FLUX）。本實作程式碼已公開於 https://github.com/SonyResearch/SPA。

English

Diffusion models typically suffer from error accumulation during iterative sampling, commonly referred to as exposure bias. We reveal systematic frequency-dependent discrepancies between training and inference, which can be interpreted as frequency-dependent SNR error. Crucially, the direction of this mismatch varies across models and timesteps, indicating that fixed correction rules do not generalize. We propose Spectral Alignment (SPA), a lightweight, guidance-based method that calibrates the power spectrum of intermediate predictions to a pre-computed prior. Our approach consists of two stages: (1) offline fitting of a parametric spectrum model from training data, and (2) inference-time guidance via efficient FFT-based gradient computation. SPA introduces minimal computational overhead (3-4\%) and is complementary to Classifier-Free Guidance (CFG). We demonstrate consistent improvements across diverse architectures, from pixel-space models (DDPM, ADM) to latent diffusion models (SD2.0, SDXL) and flow-matching models (SD3.5, FLUX). Our implementation is available at https://github.com/SonyResearch/SPA.