FlexEdit：靈活且可控的基於擴散的物件導向圖像編輯

FlexEdit: Flexible and Controllable Diffusion-based Object-centric Image Editing

March 27, 2024

作者: Trong-Tung Nguyen, Duc-Anh Nguyen, Anh Tran, Cuong Pham

cs.AI

摘要

我們的工作解決了先前在以物件為中心的編輯問題中所見到的限制，例如由於形狀差異而導致的不真實結果，以及在物件替換或插入方面的有限控制。為此，我們引入了FlexEdit，一個靈活且可控的物件編輯框架，在其中我們使用我們的FlexEdit區塊在每個去噪步驟中迭代調整潛在變數。起初，我們在測試時間優化潛在變數以符合指定的物件限制。然後，我們的框架在去噪期間自動提取適應性遮罩，以保護背景，同時將新內容無縫地融入目標圖像。我們展示了FlexEdit在各種物件編輯任務中的多功能性，並通過來自真實和合成圖像的樣本精心策劃了一個評估測試套件，以及針對以物件為中心的編輯設計的新型評估指標。我們在不同編輯場景上進行了廣泛的實驗，展示了我們的編輯框架優於最近先進的文本引導圖像編輯方法的優越性。我們的項目頁面發佈在https://flex-edit.github.io/。

English

Our work addresses limitations seen in previous approaches for object-centric editing problems, such as unrealistic results due to shape discrepancies and limited control in object replacement or insertion. To this end, we introduce FlexEdit, a flexible and controllable editing framework for objects where we iteratively adjust latents at each denoising step using our FlexEdit block. Initially, we optimize latents at test time to align with specified object constraints. Then, our framework employs an adaptive mask, automatically extracted during denoising, to protect the background while seamlessly blending new content into the target image. We demonstrate the versatility of FlexEdit in various object editing tasks and curate an evaluation test suite with samples from both real and synthetic images, along with novel evaluation metrics designed for object-centric editing. We conduct extensive experiments on different editing scenarios, demonstrating the superiority of our editing framework over recent advanced text-guided image editing methods. Our project page is published at https://flex-edit.github.io/.