GAvatar：具有隐式网格学习的可动画3D高斯化身

GAvatar: Animatable 3D Gaussian Avatars with Implicit Mesh Learning

December 18, 2023

作者: Ye Yuan, Xueting Li, Yangyi Huang, Shalini De Mello, Koki Nagano, Jan Kautz, Umar Iqbal

cs.AI

摘要

高斯飘逸成为一种强大的三维表示形式，充分利用了显式（网格）和隐式（NeRF）三维表示形式的优势。本文旨在利用高斯飘逸从文本描述中生成逼真的可动化角色，解决网格或基于NeRF的表示形式所带来的限制（例如，灵活性和效率）。然而，简单应用高斯飘逸无法生成高质量的可动化角色，并且存在学习不稳定性；它也无法捕捉精细的角色几何结构，通常导致身体部位退化。为了解决这些问题，我们首先提出了基于基元的三维高斯表示，其中高斯定义在受姿势驱动的基元内，以促进动画。其次，为了稳定和分摊数百万高斯的学习，我们建议使用神经隐式场来预测高斯属性（例如颜色）。最后，为了捕捉精细的角色几何结构并提取详细的网格，我们提出了一种基于SDF的隐式网格学习方法，用于三维高斯，该方法规范底层几何结构并提取高度详细的纹理网格。我们提出的方法GAvatar，通过仅使用文本提示实现了大规模生成多样化的可动化角色。GAvatar在外观和几何质量方面显著超越现有方法，并且在1K分辨率下实现了极快的渲染速度（100 fps）。

English

Gaussian splatting has emerged as a powerful 3D representation that harnesses the advantages of both explicit (mesh) and implicit (NeRF) 3D representations. In this paper, we seek to leverage Gaussian splatting to generate realistic animatable avatars from textual descriptions, addressing the limitations (e.g., flexibility and efficiency) imposed by mesh or NeRF-based representations. However, a naive application of Gaussian splatting cannot generate high-quality animatable avatars and suffers from learning instability; it also cannot capture fine avatar geometries and often leads to degenerate body parts. To tackle these problems, we first propose a primitive-based 3D Gaussian representation where Gaussians are defined inside pose-driven primitives to facilitate animation. Second, to stabilize and amortize the learning of millions of Gaussians, we propose to use neural implicit fields to predict the Gaussian attributes (e.g., colors). Finally, to capture fine avatar geometries and extract detailed meshes, we propose a novel SDF-based implicit mesh learning approach for 3D Gaussians that regularizes the underlying geometries and extracts highly detailed textured meshes. Our proposed method, GAvatar, enables the large-scale generation of diverse animatable avatars using only text prompts. GAvatar significantly surpasses existing methods in terms of both appearance and geometry quality, and achieves extremely fast rendering (100 fps) at 1K resolution.