利用FLAIR解決反問題

摘要

基於流模型的潛在生成模型，如Stable Diffusion 3，能夠生成質量卓越的圖像，甚至實現了逼真的文本到圖像生成。其令人印象深刻的性能表明，這些模型也應構成逆成像問題的強大先驗，但該方法尚未達到可比擬的保真度。存在幾個關鍵障礙：(i) 編碼到低維潛在空間使得基礎（正向）映射非線性；(ii) 數據似然項通常難以處理；以及(iii) 學習到的生成模型在推理過程中難以恢復罕見的、非典型的數據模式。我們提出了FLAIR，一種新穎的無訓練變分框架，利用基於流的生成模型作為逆問題的先驗。為此，我們引入了一種與退化類型無關的流匹配變分目標，並將其與確定性軌跡調整相結合，以恢復非典型模式。為了確保與觀測數據的完全一致性，我們將數據保真度和正則化項的優化分離。此外，我們引入了一種時間依賴的校準方案，其中正則化的強度根據離線精度估計進行調節。在標準成像基準上的結果表明，FLAIR在重建質量和樣本多樣性方面始終優於現有的基於擴散和流的方法。

English

Flow-based latent generative models such as Stable Diffusion 3 are able to generate images with remarkable quality, even enabling photorealistic text-to-image generation. Their impressive performance suggests that these models should also constitute powerful priors for inverse imaging problems, but that approach has not yet led to comparable fidelity. There are several key obstacles: (i) the encoding into a lower-dimensional latent space makes the underlying (forward) mapping non-linear; (ii) the data likelihood term is usually intractable; and (iii) learned generative models struggle to recover rare, atypical data modes during inference. We present FLAIR, a novel training free variational framework that leverages flow-based generative models as a prior for inverse problems. To that end, we introduce a variational objective for flow matching that is agnostic to the type of degradation, and combine it with deterministic trajectory adjustments to recover atypical modes. To enforce exact consistency with the observed data, we decouple the optimization of the data fidelity and regularization terms. Moreover, we introduce a time-dependent calibration scheme in which the strength of the regularization is modulated according to off-line accuracy estimates. Results on standard imaging benchmarks demonstrate that FLAIR consistently outperforms existing diffusion- and flow-based methods in terms of reconstruction quality and sample diversity.

利用FLAIR解決反問題

Solving Inverse Problems with FLAIR

摘要

Support