TexDreamer：朝向零樣本高保真度3D人體紋理生成

TexDreamer: Towards Zero-Shot High-Fidelity 3D Human Texture Generation

March 19, 2024

作者: Yufei Liu, Junwei Zhu, Junshu Tang, Shijie Zhang, Jiangning Zhang, Weijian Cao, Chengjie Wang, Yunsheng Wu, Dongjin Huang

cs.AI

摘要

利用語義 UV 地圖對 3D 人物進行紋理處理仍然是一個挑戰，因為獲取合理展開的 UV 面困難。儘管最近在監督多視角渲染方面取得了進展，使用大型文本到圖像（T2I）模型，但在生成速度、文本一致性和紋理質量方面仍存在問題，導致現有數據集中存在數據稀缺問題。我們提出 TexDreamer，這是第一個零樣本多模式高保真 3D 人體紋理生成模型。利用高效的紋理適應微調策略，我們將大型 T2I 模型調整為語義 UV 結構，同時保留其原始泛化能力。通過一個新穎的特徵翻譯器模組，訓練後的模型能夠在幾秒內從文本或圖像生成高保真的 3D 人體紋理。此外，我們介紹了 ArTicuLated humAn textureS（ATLAS），這是最大的高分辨率（1024 X 1024）3D 人體紋理數據集，其中包含 50k 高保真紋理與文本描述。

English

Texturing 3D humans with semantic UV maps remains a challenge due to the difficulty of acquiring reasonably unfolded UV. Despite recent text-to-3D advancements in supervising multi-view renderings using large text-to-image (T2I) models, issues persist with generation speed, text consistency, and texture quality, resulting in data scarcity among existing datasets. We present TexDreamer, the first zero-shot multimodal high-fidelity 3D human texture generation model. Utilizing an efficient texture adaptation finetuning strategy, we adapt large T2I model to a semantic UV structure while preserving its original generalization capability. Leveraging a novel feature translator module, the trained model is capable of generating high-fidelity 3D human textures from either text or image within seconds. Furthermore, we introduce ArTicuLated humAn textureS (ATLAS), the largest high-resolution (1024 X 1024) 3D human texture dataset which contains 50k high-fidelity textures with text descriptions.