在約7個步驟中進行文本引導的圖像編輯的可逆一致性蒸餾

Invertible Consistency Distillation for Text-Guided Image Editing in Around 7 Steps

June 20, 2024

作者: Nikita Starodubcev, Mikhail Khoroshikh, Artem Babenko, Dmitry Baranchuk

cs.AI

摘要

擴散蒸餾代表著一個極具前景的方向，可以在少數取樣步驟中實現忠實的文本到圖像生成。然而，儘管最近取得成功，現有的蒸餾模型仍無法提供完整的擴散能力範疇，例如實際圖像反轉，這使得許多精確的圖像操作方法成為可能。本研究旨在豐富蒸餾文本到圖像擴散模型的能力，使其能夠有效地將真實圖像編碼到其潛在空間中。為此，我們引入了可逆一致性蒸餾（iCD），這是一個通用的一致性蒸餾框架，可以在僅需3-4個推論步驟中促進高質量圖像合成和準確圖像編碼。雖然文本到圖像擴散模型的反轉問題受到高無分類器引導尺度的加劇，但我們注意到動態引導顯著降低了重構錯誤，而在生成性能上幾乎沒有明顯的降級。因此，我們證明了搭配動態引導的iCD可能作為一個非常有效的工具，用於零樣本文本引導的圖像編輯，與更昂貴的最先進替代方案競爭。

English

Diffusion distillation represents a highly promising direction for achieving faithful text-to-image generation in a few sampling steps. However, despite recent successes, existing distilled models still do not provide the full spectrum of diffusion abilities, such as real image inversion, which enables many precise image manipulation methods. This work aims to enrich distilled text-to-image diffusion models with the ability to effectively encode real images into their latent space. To this end, we introduce invertible Consistency Distillation (iCD), a generalized consistency distillation framework that facilitates both high-quality image synthesis and accurate image encoding in only 3-4 inference steps. Though the inversion problem for text-to-image diffusion models gets exacerbated by high classifier-free guidance scales, we notice that dynamic guidance significantly reduces reconstruction errors without noticeable degradation in generation performance. As a result, we demonstrate that iCD equipped with dynamic guidance may serve as a highly effective tool for zero-shot text-guided image editing, competing with more expensive state-of-the-art alternatives.