DreamMix：解耦对象属性，增强定制图像修复的可编辑性

DreamMix: Decoupling Object Attributes for Enhanced Editability in Customized Image Inpainting

November 26, 2024

作者: Yicheng Yang, Pengxiang Li, Lu Zhang, Liqian Ma, Ping Hu, Siyu Du, Yunzhi Zhuge, Xu Jia, Huchuan Lu

cs.AI

摘要

随着扩散模型的最新进展，基于主体驱动的图像修复已成为图像编辑中的热门任务。先前的方法主要侧重于保留身份，但在保持插入对象的可编辑性方面存在困难。为此，本文引入了DreamMix，一种基于扩散的生成模型，擅长将目标对象插入到给定场景中的用户指定位置，同时实现对其属性进行任意文本驱动修改。具体而言，我们利用先进的基础修复模型，并引入了一个解耦的局部-全局修复框架，以平衡精确的局部对象插入和有效的全局视觉连贯性。此外，我们提出了属性解耦机制（ADM）和文本属性替换（TAS）模块，分别改善了基于文本属性指导的多样性和区分能力。大量实验证明，DreamMix 在各种应用场景中（包括对象插入、属性编辑和小对象修复）有效平衡了身份保留和属性可编辑性。我们的代码可在 https://github.com/mycfhs/DreamMix 上公开获取。

English

Subject-driven image inpainting has emerged as a popular task in image editing alongside recent advancements in diffusion models. Previous methods primarily focus on identity preservation but struggle to maintain the editability of inserted objects. In response, this paper introduces DreamMix, a diffusion-based generative model adept at inserting target objects into given scenes at user-specified locations while concurrently enabling arbitrary text-driven modifications to their attributes. In particular, we leverage advanced foundational inpainting models and introduce a disentangled local-global inpainting framework to balance precise local object insertion with effective global visual coherence. Additionally, we propose an Attribute Decoupling Mechanism (ADM) and a Textual Attribute Substitution (TAS) module to improve the diversity and discriminative capability of the text-based attribute guidance, respectively. Extensive experiments demonstrate that DreamMix effectively balances identity preservation and attribute editability across various application scenarios, including object insertion, attribute editing, and small object inpainting. Our code is publicly available at https://github.com/mycfhs/DreamMix.