AvatarBooth：高质量和可定制的3D人类化身生成

AvatarBooth: High-Quality and Customizable 3D Human Avatar Generation

June 16, 2023

作者: Yifei Zeng, Yuanxun Lu, Xinya Ji, Yao Yao, Hao Zhu, Xun Cao

cs.AI

摘要

我们介绍了AvatarBooth，这是一种新颖的方法，可以使用文本提示或特定图像生成高质量的3D头像。与先前的方法只能根据简单的文本描述合成头像不同，我们的方法可以从随意捕捉的面部或身体图像中创建个性化头像，同时支持基于文本的模型生成和编辑。我们的关键贡献在于通过为人脸和身体分别使用双精细调整扩散模型来精确控制头像生成。这使我们能够捕捉面部外观、服装和配饰的复杂细节，从而产生高度逼真的头像生成。此外，我们引入了姿势一致性约束到优化过程中，以增强从扩散模型合成的头部图像的多视角一致性，从而消除不受控制的人体姿势的干扰。此外，我们提出了一种多分辨率渲染策略，有助于对3D头像生成进行由粗到精的监督，从而提升所提出系统的性能。生成的头像模型可以通过额外的文本描述进一步编辑，并由运动序列驱动。实验证明，AvatarBooth在从文本提示或特定图像生成方面的渲染和几何质量方面优于先前的文本到3D方法。请访问我们的项目网站https://zeng-yifei.github.io/avatarbooth_page/。

English

We introduce AvatarBooth, a novel method for generating high-quality 3D avatars using text prompts or specific images. Unlike previous approaches that can only synthesize avatars based on simple text descriptions, our method enables the creation of personalized avatars from casually captured face or body images, while still supporting text-based model generation and editing. Our key contribution is the precise avatar generation control by using dual fine-tuned diffusion models separately for the human face and body. This enables us to capture intricate details of facial appearance, clothing, and accessories, resulting in highly realistic avatar generations. Furthermore, we introduce pose-consistent constraint to the optimization process to enhance the multi-view consistency of synthesized head images from the diffusion model and thus eliminate interference from uncontrolled human poses. In addition, we present a multi-resolution rendering strategy that facilitates coarse-to-fine supervision of 3D avatar generation, thereby enhancing the performance of the proposed system. The resulting avatar model can be further edited using additional text descriptions and driven by motion sequences. Experiments show that AvatarBooth outperforms previous text-to-3D methods in terms of rendering and geometric quality from either text prompts or specific images. Please check our project website at https://zeng-yifei.github.io/avatarbooth_page/.