HeadGAP: 通過可泛化的高斯先驗生成少樣本3D頭像

HeadGAP: Few-shot 3D Head Avatar via Generalizable Gaussian Priors

August 12, 2024

作者: Xiaozheng Zheng, Chao Wen, Zhaohu Li, Weiyi Zhang, Zhuo Su, Xu Chang, Yang Zhao, Zheng Lv, Xiaoyuan Zhang, Yongjie Zhang, Guidong Wang, Lan Xu

cs.AI

摘要

本文提出了一種新穎的3D頭像創建方法，能夠從少量野外數據中進行泛化，具有高保真度和可動性的魯棒性。考慮到這個問題存在的不確定性，融入先前知識至關重要。因此，我們提出了一個包含先前學習和頭像創建階段的框架。先前學習階段利用從大規模多視角動態數據集中導出的3D頭部先驗，而頭像創建階段則應用這些先驗進行少量個性化。我們的方法通過使用基於高斯點陣的自編碼器網絡和基於部件的動態建模有效地捕捉這些先驗。我們的方法採用共享身份編碼和個性化潛在代碼，用於學習高斯基元的屬性。在頭像創建階段，我們通過利用反演和微調策略實現快速頭像個性化。大量實驗表明，我們的模型有效地利用頭部先驗，成功將其泛化到少量個性化，實現了照片般逼真的渲染質量、多視角一致性和穩定動畫。

English

In this paper, we present a novel 3D head avatar creation approach capable of generalizing from few-shot in-the-wild data with high-fidelity and animatable robustness. Given the underconstrained nature of this problem, incorporating prior knowledge is essential. Therefore, we propose a framework comprising prior learning and avatar creation phases. The prior learning phase leverages 3D head priors derived from a large-scale multi-view dynamic dataset, and the avatar creation phase applies these priors for few-shot personalization. Our approach effectively captures these priors by utilizing a Gaussian Splatting-based auto-decoder network with part-based dynamic modeling. Our method employs identity-shared encoding with personalized latent codes for individual identities to learn the attributes of Gaussian primitives. During the avatar creation phase, we achieve fast head avatar personalization by leveraging inversion and fine-tuning strategies. Extensive experiments demonstrate that our model effectively exploits head priors and successfully generalizes them to few-shot personalization, achieving photo-realistic rendering quality, multi-view consistency, and stable animation.