利用FLAIR解决逆问题

摘要

基于流的潜在生成模型，如Stable Diffusion 3，能够生成质量卓越的图像，甚至实现了逼真的文本到图像生成。其卓越性能表明，这些模型也应成为逆成像问题的强大先验，但这一方法尚未达到同等保真度。主要障碍包括：(i) 编码到低维潜在空间使得基础（正向）映射非线性；(ii) 数据似然项通常难以处理；(iii) 学习到的生成模型在推理过程中难以恢复罕见、非典型的数据模式。我们提出了FLAIR，一种无需训练的新型变分框架，它利用基于流的生成模型作为逆问题的先验。为此，我们引入了一种与退化类型无关的流匹配变分目标，并结合确定性轨迹调整以恢复非典型模式。为确保与观测数据的精确一致性，我们将数据保真度和正则化项的优化解耦。此外，我们提出了一种时间依赖的校准方案，其中正则化强度根据离线精度估计进行调节。标准成像基准测试结果表明，FLAIR在重建质量和样本多样性方面始终优于现有的基于扩散和流的方法。

English

Flow-based latent generative models such as Stable Diffusion 3 are able to generate images with remarkable quality, even enabling photorealistic text-to-image generation. Their impressive performance suggests that these models should also constitute powerful priors for inverse imaging problems, but that approach has not yet led to comparable fidelity. There are several key obstacles: (i) the encoding into a lower-dimensional latent space makes the underlying (forward) mapping non-linear; (ii) the data likelihood term is usually intractable; and (iii) learned generative models struggle to recover rare, atypical data modes during inference. We present FLAIR, a novel training free variational framework that leverages flow-based generative models as a prior for inverse problems. To that end, we introduce a variational objective for flow matching that is agnostic to the type of degradation, and combine it with deterministic trajectory adjustments to recover atypical modes. To enforce exact consistency with the observed data, we decouple the optimization of the data fidelity and regularization terms. Moreover, we introduce a time-dependent calibration scheme in which the strength of the regularization is modulated according to off-line accuracy estimates. Results on standard imaging benchmarks demonstrate that FLAIR consistently outperforms existing diffusion- and flow-based methods in terms of reconstruction quality and sample diversity.

利用FLAIR解决逆问题

Solving Inverse Problems with FLAIR

摘要

Support