GAvatar：具有隱式網格學習的可動式3D高斯頭像

GAvatar: Animatable 3D Gaussian Avatars with Implicit Mesh Learning

December 18, 2023

作者: Ye Yuan, Xueting Li, Yangyi Huang, Shalini De Mello, Koki Nagano, Jan Kautz, Umar Iqbal

cs.AI

摘要

高斯點塗已成為一種強大的3D表示法，兼具顯式（網格）和隱式（NeRF）3D表示法的優勢。本文旨在利用高斯點塗來從文字描述中生成逼真的可動化頭像，解決基於網格或NeRF表示法所帶來的限制（例如靈活性和效率）。然而，單純應用高斯點塗無法生成高質量的可動化頭像，並存在學習不穩定性；同時無法捕捉細緻的頭像幾何結構，並常導致身體部位退化。為解決這些問題，我們首先提出一種基於基元的3D高斯表示法，其中高斯定義在受姿勢驅動的基元內以促進動畫。其次，為了穩定和攤銷數百萬個高斯的學習，我們建議使用神經隱式場來預測高斯屬性（例如顏色）。最後，為了捕捉細緻的頭像幾何結構並提取詳細的網格，我們提出了一種基於SDF的隱式網格學習方法，用於3D高斯，該方法對底層幾何進行正則化並提取高度詳細的紋理網格。我們提出的方法GAvatar，使得僅通過文本提示就能大規模生成多樣的可動化頭像。GAvatar在外觀和幾何質量方面顯著超越現有方法，並實現極快的渲染速度（100 fps）在1K分辨率下。

English

Gaussian splatting has emerged as a powerful 3D representation that harnesses the advantages of both explicit (mesh) and implicit (NeRF) 3D representations. In this paper, we seek to leverage Gaussian splatting to generate realistic animatable avatars from textual descriptions, addressing the limitations (e.g., flexibility and efficiency) imposed by mesh or NeRF-based representations. However, a naive application of Gaussian splatting cannot generate high-quality animatable avatars and suffers from learning instability; it also cannot capture fine avatar geometries and often leads to degenerate body parts. To tackle these problems, we first propose a primitive-based 3D Gaussian representation where Gaussians are defined inside pose-driven primitives to facilitate animation. Second, to stabilize and amortize the learning of millions of Gaussians, we propose to use neural implicit fields to predict the Gaussian attributes (e.g., colors). Finally, to capture fine avatar geometries and extract detailed meshes, we propose a novel SDF-based implicit mesh learning approach for 3D Gaussians that regularizes the underlying geometries and extracts highly detailed textured meshes. Our proposed method, GAvatar, enables the large-scale generation of diverse animatable avatars using only text prompts. GAvatar significantly surpasses existing methods in terms of both appearance and geometry quality, and achieves extremely fast rendering (100 fps) at 1K resolution.