ByteEdit：增强、符合和加速生成图像编辑

ByteEdit: Boost, Comply and Accelerate Generative Image Editing

April 7, 2024

作者: Yuxi Ren, Jie Wu, Yanzuo Lu, Huafeng Kuang, Xin Xia, Xionghui Wang, Qianqian Wang, Yixing Zhu, Pan Xie, Shiyin Wang, Xuefeng Xiao, Yitong Wang, Min Zheng, Lean Fu

cs.AI

摘要

最近扩散式生成图像编辑的进展引发了一场深刻的革命，重塑了图像外推和修复任务的格局。尽管取得了这些进展，但该领域仍然面临固有挑战，包括：i) 质量较差；ii) 一致性差；iii) 不足的指令遵循；iv) 生成效率亚优。为了解决这些障碍，我们提出了ByteEdit，这是一个创新的反馈学习框架，经过精心设计，旨在提升、遵循和加速生成图像编辑任务。ByteEdit 无缝集成了专注于提升美学和图像文本对齐的图像奖励模型，同时引入了一个针对促进输出一致性的密集像素级奖励模型。此外，我们提出了一种开创性的对抗性和渐进式反馈学习策略，以加快模型的推断速度。通过广泛的大规模用户评估，我们展示了ByteEdit 在生成质量和一致性方面均超越了领先的生成图像编辑产品，包括 Adobe、Canva 和美图。与基准模型相比，ByteEdit-Outpainting 在质量和一致性方面分别提升了 388% 和 135%。实验证明，我们的加速模型在质量和一致性方面保持了出色的性能结果。

English

Recent advancements in diffusion-based generative image editing have sparked a profound revolution, reshaping the landscape of image outpainting and inpainting tasks. Despite these strides, the field grapples with inherent challenges, including: i) inferior quality; ii) poor consistency; iii) insufficient instrcution adherence; iv) suboptimal generation efficiency. To address these obstacles, we present ByteEdit, an innovative feedback learning framework meticulously designed to Boost, Comply, and Accelerate Generative Image Editing tasks. ByteEdit seamlessly integrates image reward models dedicated to enhancing aesthetics and image-text alignment, while also introducing a dense, pixel-level reward model tailored to foster coherence in the output. Furthermore, we propose a pioneering adversarial and progressive feedback learning strategy to expedite the model's inference speed. Through extensive large-scale user evaluations, we demonstrate that ByteEdit surpasses leading generative image editing products, including Adobe, Canva, and MeiTu, in both generation quality and consistency. ByteEdit-Outpainting exhibits a remarkable enhancement of 388% and 135% in quality and consistency, respectively, when compared to the baseline model. Experiments also verfied that our acceleration models maintains excellent performance results in terms of quality and consistency.