D-Flow:通过流进行控制生成的微分。
D-Flow: Differentiating through Flows for Controlled Generation
February 21, 2024
作者: Heli Ben-Hamu, Omri Puny, Itai Gat, Brian Karrer, Uriel Singer, Yaron Lipman
cs.AI
摘要
在无需重新训练特定任务模型的情况下,控制最先进的扩散和流匹配(FM)模型的生成结果,为解决逆问题、有条件生成以及一般受控生成提供了强大工具。在这项工作中,我们介绍了D-Flow,这是一个简单的框架,通过对流进行微分,优化源(噪声)点来控制生成过程。我们通过关键观察来推动这一框架,即针对使用高斯概率路径训练的扩散/FM模型,通过生成过程进行微分会将梯度投影到数据流形上,从而将先验隐式注入到优化过程中。我们在线性和非线性受控生成问题上验证了我们的框架,包括图像和音频逆问题以及有条件的分子生成,实现了领先水平的性能。
English
Taming the generation outcome of state of the art Diffusion and Flow-Matching
(FM) models without having to re-train a task-specific model unlocks a powerful
tool for solving inverse problems, conditional generation, and controlled
generation in general. In this work we introduce D-Flow, a simple framework for
controlling the generation process by differentiating through the flow,
optimizing for the source (noise) point. We motivate this framework by our key
observation stating that for Diffusion/FM models trained with Gaussian
probability paths, differentiating through the generation process projects
gradient on the data manifold, implicitly injecting the prior into the
optimization process. We validate our framework on linear and non-linear
controlled generation problems including: image and audio inverse problems and
conditional molecule generation reaching state of the art performance across
all.