AniFaceDrawing：在您的素描过程中探索动漫肖像

AniFaceDrawing: Anime Portrait Exploration during Your Sketching

June 13, 2023

作者: Zhengyu Huang, Haoran Xie, Tsukasa Fukusato, Kazunori Miyata

cs.AI

摘要

本文关注人工智能（AI）如何辅助用户在创作动漫肖像时将粗糙草图转换为动漫肖像的过程。输入是一系列逐渐完善的手绘草图，输出是一系列与输入草图对应的高质量动漫肖像，作为指导。尽管最近的生成对抗网络（GANs）可以生成高质量图像，但由于条件图像生成中存在的问题，从完成度较低的草图生成高质量图像是一个具有挑战性的问题。即使使用最新的草图到图像（S2I）技术，对于动漫肖像，由于动漫风格往往比写实风格更抽象，仍然难以从不完整的粗糙草图中创建高质量图像。为了解决这个问题，我们采用了StyleGAN的潜在空间探索和两阶段训练策略。我们认为手绘草图的输入笔画对应于StyleGAN的潜在结构编码中与边缘信息相关的属性，并将笔画与这些属性之间的匹配称为笔画级解缠。在第一阶段，我们使用预训练的StyleGAN模型作为教师编码器训练了一个图像编码器。在第二阶段，我们模拟了生成图像的绘制过程，无需任何额外数据（标签），并训练了用于生成高质量肖像图像的草图编码器，使其特征与教师编码器中解缠表示对齐。我们通过定性和定量评估验证了所提出的渐进式S2I系统，并成功从不完整的渐进草图中生成了高质量动漫肖像。我们的用户研究证明了该系统在动漫风格艺术创作辅助方面的有效性。

English

In this paper, we focus on how artificial intelligence (AI) can be used to assist users in the creation of anime portraits, that is, converting rough sketches into anime portraits during their sketching process. The input is a sequence of incomplete freehand sketches that are gradually refined stroke by stroke, while the output is a sequence of high-quality anime portraits that correspond to the input sketches as guidance. Although recent GANs can generate high quality images, it is a challenging problem to maintain the high quality of generated images from sketches with a low degree of completion due to ill-posed problems in conditional image generation. Even with the latest sketch-to-image (S2I) technology, it is still difficult to create high-quality images from incomplete rough sketches for anime portraits since anime style tend to be more abstract than in realistic style. To address this issue, we adopt a latent space exploration of StyleGAN with a two-stage training strategy. We consider the input strokes of a freehand sketch to correspond to edge information-related attributes in the latent structural code of StyleGAN, and term the matching between strokes and these attributes stroke-level disentanglement. In the first stage, we trained an image encoder with the pre-trained StyleGAN model as a teacher encoder. In the second stage, we simulated the drawing process of the generated images without any additional data (labels) and trained the sketch encoder for incomplete progressive sketches to generate high-quality portrait images with feature alignment to the disentangled representations in the teacher encoder. We verified the proposed progressive S2I system with both qualitative and quantitative evaluations and achieved high-quality anime portraits from incomplete progressive sketches. Our user study proved its effectiveness in art creation assistance for the anime style.