FitMe：深度逼真的3D可塑建模头像

摘要

本文介绍了FitMe，一种面部反射模型和可微分渲染优化流程，可用于从单个或多个图像中获取高保真可渲染的人体化身。该模型由多模态风格生成器组成，以扩散和镜面反射为基础捕捉面部外观，并包含基于PCA的形状模型。我们采用快速可微分渲染过程，可用于优化流程，同时实现逼真的面部着色。我们的优化过程通过利用基于风格的潜在表示和形状模型的表现力，准确捕捉高细节的面部反射和形状。FitMe在单个“野外”面部图像上实现了最先进的反射获取和身份保留，同时在提供多个与同一身份相关的无约束面部图像时产生令人印象深刻的扫描式结果。与最近的隐式化身重建相比，FitMe仅需一分钟即可生成可重新照明的基于网格和纹理的化身，可供最终用户应用程序使用。

English

In this paper, we introduce FitMe, a facial reflectance model and a differentiable rendering optimization pipeline, that can be used to acquire high-fidelity renderable human avatars from single or multiple images. The model consists of a multi-modal style-based generator, that captures facial appearance in terms of diffuse and specular reflectance, and a PCA-based shape model. We employ a fast differentiable rendering process that can be used in an optimization pipeline, while also achieving photorealistic facial shading. Our optimization process accurately captures both the facial reflectance and shape in high-detail, by exploiting the expressivity of the style-based latent representation and of our shape model. FitMe achieves state-of-the-art reflectance acquisition and identity preservation on single "in-the-wild" facial images, while it produces impressive scan-like results, when given multiple unconstrained facial images pertaining to the same identity. In contrast with recent implicit avatar reconstructions, FitMe requires only one minute and produces relightable mesh and texture-based avatars, that can be used by end-user applications.

FitMe：深度逼真的3D可塑建模头像

FitMe: Deep Photorealistic 3D Morphable Model Avatars

摘要

Support