TexDreamer：朝向零样本高保真度3D人体纹理生成

TexDreamer: Towards Zero-Shot High-Fidelity 3D Human Texture Generation

March 19, 2024

作者: Yufei Liu, Junwei Zhu, Junshu Tang, Shijie Zhang, Jiangning Zhang, Weijian Cao, Chengjie Wang, Yunsheng Wu, Dongjin Huang

cs.AI

摘要

利用语义UV映射为3D人体进行纹理处理仍然是一个挑战，这是因为获取合理展开的UV的困难。尽管最近在使用大型文本到图像（T2I）模型监督多视角渲染方面取得了进展，但在生成速度、文本一致性和纹理质量方面仍然存在问题，导致现有数据集中存在数据稀缺。我们提出了TexDreamer，这是第一个零样本多模态高保真度3D人体纹理生成模型。利用高效的纹理适应微调策略，我们将大型T2I模型调整到语义UV结构，同时保留其原始的泛化能力。通过利用一种新颖的特征转换器模块，训练好的模型能够在几秒内从文本或图像生成高保真度的3D人体纹理。此外，我们介绍了ArTicuLated humAn textureS（ATLAS），这是最大的高分辨率（1024 X 1024）3D人体纹理数据集，包含了50k个带有文本描述的高保真度纹理。

English

Texturing 3D humans with semantic UV maps remains a challenge due to the difficulty of acquiring reasonably unfolded UV. Despite recent text-to-3D advancements in supervising multi-view renderings using large text-to-image (T2I) models, issues persist with generation speed, text consistency, and texture quality, resulting in data scarcity among existing datasets. We present TexDreamer, the first zero-shot multimodal high-fidelity 3D human texture generation model. Utilizing an efficient texture adaptation finetuning strategy, we adapt large T2I model to a semantic UV structure while preserving its original generalization capability. Leveraging a novel feature translator module, the trained model is capable of generating high-fidelity 3D human textures from either text or image within seconds. Furthermore, we introduce ArTicuLated humAn textureS (ATLAS), the largest high-resolution (1024 X 1024) 3D human texture dataset which contains 50k high-fidelity textures with text descriptions.