細節至關重要：用於細節豐富的StyleGAN反轉和高品質圖像編輯的StyleFeatureEditor

The Devil is in the Details: StyleFeatureEditor for Detail-Rich StyleGAN Inversion and High Quality Image Editing

June 15, 2024

作者: Denis Bobkov, Vadim Titov, Aibek Alanov, Dmitry Vetrov

cs.AI

摘要

透過 StyleGAN 逆向操作來操控真實圖像屬性的任務已經得到廣泛研究。這個過程涉及從經過良好訓練的 StyleGAN 生成器中搜索潛在變數，這些變數可以合成一幅真實圖像，修改這些潛在變數，然後合成一幅具有所需編輯的圖像。必須在重建品質和編輯能力之間取得平衡。早期的研究利用低維度的 W 空間進行潛在搜索，這有助於有效編輯，但在重建精細細節方面遇到困難。最近的研究轉向高維度的特徵空間 F，成功地逆向輸入圖像，但在編輯過程中失去了許多細節。在本文中，我們介紹了 StyleFeatureEditor -- 一種新穎的方法，可以在 w-latents 和 F-latents 中進行編輯。這種技術不僅允許重建更細微的圖像細節，還確保在編輯過程中保留這些細節。我們還提出了一個新的訓練流程，專門設計來訓練我們的模型以準確編輯 F-latents。我們的方法與最先進的編碼方法進行了比較，表明我們的模型在重建品質方面表現優異，能夠編輯即使是具有挑戰性的跨領域示例。代碼可在 https://github.com/AIRI-Institute/StyleFeatureEditor 找到。

English

The task of manipulating real image attributes through StyleGAN inversion has been extensively researched. This process involves searching latent variables from a well-trained StyleGAN generator that can synthesize a real image, modifying these latent variables, and then synthesizing an image with the desired edits. A balance must be struck between the quality of the reconstruction and the ability to edit. Earlier studies utilized the low-dimensional W-space for latent search, which facilitated effective editing but struggled with reconstructing intricate details. More recent research has turned to the high-dimensional feature space F, which successfully inverses the input image but loses much of the detail during editing. In this paper, we introduce StyleFeatureEditor -- a novel method that enables editing in both w-latents and F-latents. This technique not only allows for the reconstruction of finer image details but also ensures their preservation during editing. We also present a new training pipeline specifically designed to train our model to accurately edit F-latents. Our method is compared with state-of-the-art encoding approaches, demonstrating that our model excels in terms of reconstruction quality and is capable of editing even challenging out-of-domain examples. Code is available at https://github.com/AIRI-Institute/StyleFeatureEditor.