AvatarBooth：高品質且可定制的3D人類化身生成

AvatarBooth: High-Quality and Customizable 3D Human Avatar Generation

June 16, 2023

作者: Yifei Zeng, Yuanxun Lu, Xinya Ji, Yao Yao, Hao Zhu, Xun Cao

cs.AI

摘要

我們介紹了AvatarBooth，一種新穎的方法，可使用文字提示或特定圖像生成高質量的3D頭像。與先前只能根據簡單文字描述合成頭像的方法不同，我們的方法可以從隨意捕捉的面部或身體圖像創建個性化頭像，同時支持基於文本的模型生成和編輯。我們的主要貢獻在於使用為人臉和身體分別進行精細調整的雙擴散模型來精確控制頭像生成。這使我們能夠捕捉面部外觀、服裝和配飾的細微細節，從而產生高度逼真的頭像生成。此外，我們引入了姿勢一致性約束來增強優化過程中從擴散模型合成的頭部圖像的多視角一致性，從而消除來自不受控制的人體姿勢的干擾。此外，我們提出了一種多分辨率渲染策略，有助於粗到細監督3D頭像生成，從而提高所提出系統的性能。生成的頭像模型可以使用額外的文本描述進行進一步編輯，並由運動序列驅動。實驗表明，AvatarBooth在從文字提示或特定圖像生成方面在渲染和幾何質量上優於先前的文字轉3D方法。請查看我們的項目網站：https://zeng-yifei.github.io/avatarbooth_page/。

English

We introduce AvatarBooth, a novel method for generating high-quality 3D avatars using text prompts or specific images. Unlike previous approaches that can only synthesize avatars based on simple text descriptions, our method enables the creation of personalized avatars from casually captured face or body images, while still supporting text-based model generation and editing. Our key contribution is the precise avatar generation control by using dual fine-tuned diffusion models separately for the human face and body. This enables us to capture intricate details of facial appearance, clothing, and accessories, resulting in highly realistic avatar generations. Furthermore, we introduce pose-consistent constraint to the optimization process to enhance the multi-view consistency of synthesized head images from the diffusion model and thus eliminate interference from uncontrolled human poses. In addition, we present a multi-resolution rendering strategy that facilitates coarse-to-fine supervision of 3D avatar generation, thereby enhancing the performance of the proposed system. The resulting avatar model can be further edited using additional text descriptions and driven by motion sequences. Experiments show that AvatarBooth outperforms previous text-to-3D methods in terms of rendering and geometric quality from either text prompts or specific images. Please check our project website at https://zeng-yifei.github.io/avatarbooth_page/.