HAAR：文本条件生成模型用于三维基于发丝的人类发型

HAAR: Text-Conditioned Generative Model of 3D Strand-based Human Hairstyles

December 18, 2023

作者: Vanessa Sklyarova, Egor Zakharov, Otmar Hilliges, Michael J. Black, Justus Thies

cs.AI

摘要

我们提出了HAAR，这是一种基于发束的新型三维人类发型生成模型。具体来说，基于文本输入，HAAR生成可用作现代计算机图形引擎中生产级资产的三维发型。当前基于人工智能的生成模型利用强大的二维先验来重建以点云、网格或体积函数形式呈现的三维内容。然而，通过使用二维先验，它们固有地仅限于恢复视觉部分。高度遮挡的发型结构无法用这些方法重建，它们仅模拟“外壳”，这不适用于基于物理的渲染或仿真流程。相比之下，我们提出了一种首个文本引导的生成方法，它使用三维发束作为基础表示。利用二维视觉问答（VQA）系统，我们自动注释从一小组艺术家创建的发型中生成的合成发型模型。这使我们能够训练在常见发型UV空间中运行的潜在扩散模型。通过定性和定量研究，我们展示了所提出模型的能力，并将其与现有发型生成方法进行了比较。

English

We present HAAR, a new strand-based generative model for 3D human hairstyles. Specifically, based on textual inputs, HAAR produces 3D hairstyles that could be used as production-level assets in modern computer graphics engines. Current AI-based generative models take advantage of powerful 2D priors to reconstruct 3D content in the form of point clouds, meshes, or volumetric functions. However, by using the 2D priors, they are intrinsically limited to only recovering the visual parts. Highly occluded hair structures can not be reconstructed with those methods, and they only model the ''outer shell'', which is not ready to be used in physics-based rendering or simulation pipelines. In contrast, we propose a first text-guided generative method that uses 3D hair strands as an underlying representation. Leveraging 2D visual question-answering (VQA) systems, we automatically annotate synthetic hair models that are generated from a small set of artist-created hairstyles. This allows us to train a latent diffusion model that operates in a common hairstyle UV space. In qualitative and quantitative studies, we demonstrate the capabilities of the proposed model and compare it to existing hairstyle generation approaches.