AniFaceDrawing：在您的素描過程中探索動畫肖像

AniFaceDrawing: Anime Portrait Exploration during Your Sketching

June 13, 2023

作者: Zhengyu Huang, Haoran Xie, Tsukasa Fukusato, Kazunori Miyata

cs.AI

摘要

本文著重探討如何利用人工智慧（AI）來協助使用者創作動漫肖像，即在他們素描過程中將粗略素描轉換為動漫肖像。輸入是一系列逐漸由筆劃精煉的不完整自由手繪素描，而輸出是一系列高質量的動漫肖像，以輸入素描作為指導。儘管最近的生成對抗網絡（GANs）可以產生高質量圖像，但由於條件圖像生成中的棘手問題，從具有低完成度的素描生成高質量圖像仍然是一個具有挑戰性的問題。即使使用最新的素描轉圖像（S2I）技術，對於動漫肖像的不完整粗略素描生成高質量圖像仍然困難，因為動漫風格往往比現實風格更抽象。為了解決這個問題，我們採用了StyleGAN的潛在空間探索和雙階段訓練策略。我們認為自由手繪素描的輸入筆劃對應於StyleGAN的潛在結構代碼中與邊緣信息相關的屬性，並將筆劃與這些屬性之間的匹配稱為筆劃級別的解耦。在第一階段，我們使用預訓練的StyleGAN模型作為教師編碼器來訓練圖像編碼器。在第二階段，我們模擬生成圖像的繪製過程，無需任何額外數據（標籤），並訓練不完整漸進素描的素描編碼器，以便生成具有與教師編碼器中解耦表示相符的特徵對齊的高質量肖像。我們通過定性和定量評估驗證了所提出的漸進S2I系統，並從不完整的漸進素描中獲得了高質量的動漫肖像。我們的使用者研究證明了它在動漫風格藝術創作協助中的有效性。

English

In this paper, we focus on how artificial intelligence (AI) can be used to assist users in the creation of anime portraits, that is, converting rough sketches into anime portraits during their sketching process. The input is a sequence of incomplete freehand sketches that are gradually refined stroke by stroke, while the output is a sequence of high-quality anime portraits that correspond to the input sketches as guidance. Although recent GANs can generate high quality images, it is a challenging problem to maintain the high quality of generated images from sketches with a low degree of completion due to ill-posed problems in conditional image generation. Even with the latest sketch-to-image (S2I) technology, it is still difficult to create high-quality images from incomplete rough sketches for anime portraits since anime style tend to be more abstract than in realistic style. To address this issue, we adopt a latent space exploration of StyleGAN with a two-stage training strategy. We consider the input strokes of a freehand sketch to correspond to edge information-related attributes in the latent structural code of StyleGAN, and term the matching between strokes and these attributes stroke-level disentanglement. In the first stage, we trained an image encoder with the pre-trained StyleGAN model as a teacher encoder. In the second stage, we simulated the drawing process of the generated images without any additional data (labels) and trained the sketch encoder for incomplete progressive sketches to generate high-quality portrait images with feature alignment to the disentangled representations in the teacher encoder. We verified the proposed progressive S2I system with both qualitative and quantitative evaluations and achieved high-quality anime portraits from incomplete progressive sketches. Our user study proved its effectiveness in art creation assistance for the anime style.