D-Flow：通過流動進行控制生成

D-Flow: Differentiating through Flows for Controlled Generation

February 21, 2024

作者: Heli Ben-Hamu, Omri Puny, Itai Gat, Brian Karrer, Uriel Singer, Yaron Lipman

cs.AI

摘要

在無需重新訓練特定任務模型的情況下，控制最先進的擴散和流匹配（FM）模型生成結果的能力，為解決反問題、有條件生成以及一般控制生成提供了一個強大的工具。在這項工作中，我們介紹了D-Flow，一個簡單的框架，通過對流進行微分，優化源（噪聲）點來控制生成過程。我們通過關鍵觀察來推動這個框架，該觀察指出，對於使用高斯概率路徑訓練的擴散/FM模型，通過生成過程進行微分會將梯度投影到數據流形上，從而將先驗隱式注入到優化過程中。我們在線性和非線性控制生成問題上驗證了我們的框架，包括：圖像和音頻反問題以及有條件的分子生成，在所有問題上均達到了最先進的性能。

English

Taming the generation outcome of state of the art Diffusion and Flow-Matching (FM) models without having to re-train a task-specific model unlocks a powerful tool for solving inverse problems, conditional generation, and controlled generation in general. In this work we introduce D-Flow, a simple framework for controlling the generation process by differentiating through the flow, optimizing for the source (noise) point. We motivate this framework by our key observation stating that for Diffusion/FM models trained with Gaussian probability paths, differentiating through the generation process projects gradient on the data manifold, implicitly injecting the prior into the optimization process. We validate our framework on linear and non-linear controlled generation problems including: image and audio inverse problems and conditional molecule generation reaching state of the art performance across all.