D-Flow：通过流进行控制生成的微分。

D-Flow: Differentiating through Flows for Controlled Generation

February 21, 2024

作者: Heli Ben-Hamu, Omri Puny, Itai Gat, Brian Karrer, Uriel Singer, Yaron Lipman

cs.AI

摘要

在无需重新训练特定任务模型的情况下，控制最先进的扩散和流匹配（FM）模型的生成结果，为解决逆问题、有条件生成以及一般受控生成提供了强大工具。在这项工作中，我们介绍了D-Flow，这是一个简单的框架，通过对流进行微分，优化源（噪声）点来控制生成过程。我们通过关键观察来推动这一框架，即针对使用高斯概率路径训练的扩散/FM模型，通过生成过程进行微分会将梯度投影到数据流形上，从而将先验隐式注入到优化过程中。我们在线性和非线性受控生成问题上验证了我们的框架，包括图像和音频逆问题以及有条件的分子生成，实现了领先水平的性能。

English

Taming the generation outcome of state of the art Diffusion and Flow-Matching (FM) models without having to re-train a task-specific model unlocks a powerful tool for solving inverse problems, conditional generation, and controlled generation in general. In this work we introduce D-Flow, a simple framework for controlling the generation process by differentiating through the flow, optimizing for the source (noise) point. We motivate this framework by our key observation stating that for Diffusion/FM models trained with Gaussian probability paths, differentiating through the generation process projects gradient on the data manifold, implicitly injecting the prior into the optimization process. We validate our framework on linear and non-linear controlled generation problems including: image and audio inverse problems and conditional molecule generation reaching state of the art performance across all.