细节决定成败：用于细节丰富的StyleGAN反演和高质量图像编辑的StyleFeatureEditor

The Devil is in the Details: StyleFeatureEditor for Detail-Rich StyleGAN Inversion and High Quality Image Editing

June 15, 2024

作者: Denis Bobkov, Vadim Titov, Aibek Alanov, Dmitry Vetrov

cs.AI

摘要

通过StyleGAN反演来操纵真实图像属性的任务已经得到广泛研究。这个过程涉及从经过良好训练的StyleGAN生成器中搜索潜在变量，这些变量可以合成真实图像，修改这些潜在变量，然后合成具有所需编辑的图像。必须在重建质量和编辑能力之间取得平衡。早期研究利用低维度的W空间进行潜在搜索，这有助于有效编辑，但在重建复杂细节方面存在困难。最近的研究转向高维特征空间F，成功地反演输入图像，但在编辑过程中丢失了许多细节。在本文中，我们介绍了StyleFeatureEditor -- 一种新颖的方法，可以在w-latents和F-latents中进行编辑。这种技术不仅允许重建更精细的图像细节，还确保在编辑过程中保留这些细节。我们还提出了一种新的训练流程，专门设计用于训练我们的模型准确编辑F-latents。我们的方法与最先进的编码方法进行了比较，表明我们的模型在重建质量方面表现优异，并且能够编辑甚至具有挑战性的跨领域示例。代码可在https://github.com/AIRI-Institute/StyleFeatureEditor获得。

English

The task of manipulating real image attributes through StyleGAN inversion has been extensively researched. This process involves searching latent variables from a well-trained StyleGAN generator that can synthesize a real image, modifying these latent variables, and then synthesizing an image with the desired edits. A balance must be struck between the quality of the reconstruction and the ability to edit. Earlier studies utilized the low-dimensional W-space for latent search, which facilitated effective editing but struggled with reconstructing intricate details. More recent research has turned to the high-dimensional feature space F, which successfully inverses the input image but loses much of the detail during editing. In this paper, we introduce StyleFeatureEditor -- a novel method that enables editing in both w-latents and F-latents. This technique not only allows for the reconstruction of finer image details but also ensures their preservation during editing. We also present a new training pipeline specifically designed to train our model to accurately edit F-latents. Our method is compared with state-of-the-art encoding approaches, demonstrating that our model excels in terms of reconstruction quality and is capable of editing even challenging out-of-domain examples. Code is available at https://github.com/AIRI-Institute/StyleFeatureEditor.