FlexEdit：灵活可控的基于扩散的以对象为中心的图像编辑

FlexEdit: Flexible and Controllable Diffusion-based Object-centric Image Editing

March 27, 2024

作者: Trong-Tung Nguyen, Duc-Anh Nguyen, Anh Tran, Cuong Pham

cs.AI

摘要

我们的工作解决了以往针对以物体为中心的编辑问题所存在的局限性，例如由于形状差异导致的不真实结果以及在物体替换或插入中受限的控制。为此，我们引入了FlexEdit，这是一个灵活且可控的物体编辑框架，我们在每个去噪步骤中使用我们的FlexEdit块迭代调整潜变量。最初，我们在测试时间优化潜变量以与指定的物体约束对齐。然后，我们的框架采用自适应蒙版，在去噪过程中自动提取，以保护背景并将新内容无缝融合到目标图像中。我们展示了FlexEdit在各种物体编辑任务中的多功能性，并策划了一个包含真实和合成图像样本的评估测试套件，以及专为以物体为中心的编辑设计的新型评估指标。我们在不同编辑场景上进行了大量实验，展示了我们的编辑框架相对于最新的文本引导图像编辑方法的优越性。我们的项目页面发布在https://flex-edit.github.io/。

English

Our work addresses limitations seen in previous approaches for object-centric editing problems, such as unrealistic results due to shape discrepancies and limited control in object replacement or insertion. To this end, we introduce FlexEdit, a flexible and controllable editing framework for objects where we iteratively adjust latents at each denoising step using our FlexEdit block. Initially, we optimize latents at test time to align with specified object constraints. Then, our framework employs an adaptive mask, automatically extracted during denoising, to protect the background while seamlessly blending new content into the target image. We demonstrate the versatility of FlexEdit in various object editing tasks and curate an evaluation test suite with samples from both real and synthetic images, along with novel evaluation metrics designed for object-centric editing. We conduct extensive experiments on different editing scenarios, demonstrating the superiority of our editing framework over recent advanced text-guided image editing methods. Our project page is published at https://flex-edit.github.io/.