TurboEdit：即时文本图像编辑

TurboEdit: Instant text-based image editing

August 14, 2024

作者: Zongze Wu, Nicholas Kolkin, Jonathan Brandt, Richard Zhang, Eli Shechtman

cs.AI

摘要

在少步扩散模型的背景下，我们解决了精确图像反演和解耦图像编辑的挑战。我们引入了一种基于编码器的迭代反演技术。反演网络以输入图像和前一步重建图像为条件，从而使下一次重建朝向输入图像进行校正。我们展示了在少步扩散模型中，通过以（自动生成的）详细文本提示为条件，可以轻松实现解耦控制。为了操纵反演图像，我们固定噪声图并修改文本提示中的一个属性（可以手动或通过基于LLM驱动的指令编辑），从而生成一个类似于输入图像但只改变一个属性的新图像。它还可以控制编辑强度并接受指导性文本提示。我们的方法实现了实时逼真的文本引导图像编辑，仅需要8次反演中的功能评估（一次性成本）和每次编辑4次功能评估。我们的方法不仅速度快，而且在多步扩散编辑技术方面表现显著优越。

English

We address the challenges of precise image inversion and disentangled image editing in the context of few-step diffusion models. We introduce an encoder based iterative inversion technique. The inversion network is conditioned on the input image and the reconstructed image from the previous step, allowing for correction of the next reconstruction towards the input image. We demonstrate that disentangled controls can be easily achieved in the few-step diffusion model by conditioning on an (automatically generated) detailed text prompt. To manipulate the inverted image, we freeze the noise maps and modify one attribute in the text prompt (either manually or via instruction based editing driven by an LLM), resulting in the generation of a new image similar to the input image with only one attribute changed. It can further control the editing strength and accept instructive text prompt. Our approach facilitates realistic text-guided image edits in real-time, requiring only 8 number of functional evaluations (NFEs) in inversion (one-time cost) and 4 NFEs per edit. Our method is not only fast, but also significantly outperforms state-of-the-art multi-step diffusion editing techniques.