HeadGAP：通过通用高斯先验实现少样本3D头像生成

HeadGAP: Few-shot 3D Head Avatar via Generalizable Gaussian Priors

August 12, 2024

作者: Xiaozheng Zheng, Chao Wen, Zhaohu Li, Weiyi Zhang, Zhuo Su, Xu Chang, Yang Zhao, Zheng Lv, Xiaoyuan Zhang, Yongjie Zhang, Guidong Wang, Lan Xu

cs.AI

摘要

本文提出了一种新颖的3D头像创建方法，能够从少量野外数据中实现高保真度和可动画鲁棒性的泛化。考虑到这一问题的不确定性，融入先验知识至关重要。因此，我们提出了一个包括先验学习和头像创建阶段的框架。先验学习阶段利用从大规模多视角动态数据集中得出的3D头部先验，头像创建阶段则应用这些先验进行少样本个性化。我们的方法通过利用基于高斯散射的自动解码器网络和基于部件的动态建模有效地捕捉这些先验。我们的方法采用共享身份编码和个性化潜在代码，用于学习高斯基元的属性。在头像创建阶段，我们通过倒置和微调策略实现了快速头像个性化。大量实验证明，我们的模型有效地利用头部先验，并成功将其泛化到少样本个性化，实现了照片级渲染质量、多视角一致性和稳定动画。

English

In this paper, we present a novel 3D head avatar creation approach capable of generalizing from few-shot in-the-wild data with high-fidelity and animatable robustness. Given the underconstrained nature of this problem, incorporating prior knowledge is essential. Therefore, we propose a framework comprising prior learning and avatar creation phases. The prior learning phase leverages 3D head priors derived from a large-scale multi-view dynamic dataset, and the avatar creation phase applies these priors for few-shot personalization. Our approach effectively captures these priors by utilizing a Gaussian Splatting-based auto-decoder network with part-based dynamic modeling. Our method employs identity-shared encoding with personalized latent codes for individual identities to learn the attributes of Gaussian primitives. During the avatar creation phase, we achieve fast head avatar personalization by leveraging inversion and fine-tuning strategies. Extensive experiments demonstrate that our model effectively exploits head priors and successfully generalizes them to few-shot personalization, achieving photo-realistic rendering quality, multi-view consistency, and stable animation.