時間對齊引導：擴散模型中的流形採樣

摘要

擴散模型作為生成模型已取得顯著成功。然而，即便是訓練良好的模型，在生成過程中也可能累積誤差。這些誤差在應用任意引導以將樣本導向期望屬性時尤為棘手，因為這往往會破壞樣本的保真度。本文提出了一種通用解決方案，以應對擴散模型中觀察到的離流形現象。我們的方法利用時間預測器來估計每個時間步與期望數據流形的偏差，發現較大的時間間隔與生成質量下降相關。隨後，我們設計了一種新穎的引導機制——「時間對齊引導」（TAG），在生成過程中的每個時間步將樣本吸引回期望的流形。通過大量實驗，我們證明TAG能夠在每個時間步持續生成與期望流形緊密對齊的樣本，從而在各種下游任務中顯著提升生成質量。

English

Diffusion models have achieved remarkable success as generative models. However, even a well-trained model can accumulate errors throughout the generation process. These errors become particularly problematic when arbitrary guidance is applied to steer samples toward desired properties, which often breaks sample fidelity. In this paper, we propose a general solution to address the off-manifold phenomenon observed in diffusion models. Our approach leverages a time predictor to estimate deviations from the desired data manifold at each timestep, identifying that a larger time gap is associated with reduced generation quality. We then design a novel guidance mechanism, `Temporal Alignment Guidance' (TAG), attracting the samples back to the desired manifold at every timestep during generation. Through extensive experiments, we demonstrate that TAG consistently produces samples closely aligned with the desired manifold at each timestep, leading to significant improvements in generation quality across various downstream tasks.

時間對齊引導：擴散模型中的流形採樣

Temporal Alignment Guidance: On-Manifold Sampling in Diffusion Models

摘要

Support