ByteEdit：增強、符合和加速生成式圖像編輯

摘要

最近在基於擴散的生成式圖像編輯方面取得的進展引發了一場深刻的革命，重塑了圖像外部繪製和內部修補任務的格局。儘管取得了這些進展，但該領域仍面臨著固有挑戰，包括：i) 質量較差；ii) 一致性差；iii) 不足的指導遵循；iv) 生成效率亞優。為了應對這些障礙，我們提出了ByteEdit，一個精心設計的創新反饋學習框架，旨在提升、遵循和加速生成式圖像編輯任務。ByteEdit巧妙地整合了專注於提升美學和圖像-文本對齊的圖像獎勵模型，同時引入了一個針對促進輸出一致性而量身定制的密集像素級獎勵模型。此外，我們提出了一種開創性的對抗性和漸進式反饋學習策略，以加快模型的推理速度。通過大規模用戶評估，我們展示了ByteEdit在生成質量和一致性方面均超越了領先的生成式圖像編輯產品，包括Adobe、Canva和MeiTu。與基準模型相比，ByteEdit-Outpainting在質量和一致性方面分別顯著提高了388%和135%。實驗還證實，我們的加速模型在質量和一致性方面保持了出色的性能結果。

English

Recent advancements in diffusion-based generative image editing have sparked a profound revolution, reshaping the landscape of image outpainting and inpainting tasks. Despite these strides, the field grapples with inherent challenges, including: i) inferior quality; ii) poor consistency; iii) insufficient instrcution adherence; iv) suboptimal generation efficiency. To address these obstacles, we present ByteEdit, an innovative feedback learning framework meticulously designed to Boost, Comply, and Accelerate Generative Image Editing tasks. ByteEdit seamlessly integrates image reward models dedicated to enhancing aesthetics and image-text alignment, while also introducing a dense, pixel-level reward model tailored to foster coherence in the output. Furthermore, we propose a pioneering adversarial and progressive feedback learning strategy to expedite the model's inference speed. Through extensive large-scale user evaluations, we demonstrate that ByteEdit surpasses leading generative image editing products, including Adobe, Canva, and MeiTu, in both generation quality and consistency. ByteEdit-Outpainting exhibits a remarkable enhancement of 388% and 135% in quality and consistency, respectively, when compared to the baseline model. Experiments also verfied that our acceleration models maintains excellent performance results in terms of quality and consistency.

ByteEdit：增強、符合和加速生成式圖像編輯

ByteEdit: Boost, Comply and Accelerate Generative Image Editing

摘要

Summary

Support