TEXGen：網格紋理生成式擴散模型

摘要

雖然高品質紋理貼圖對於實現逼真的3D資產渲染至關重要，但現有研究極少探索直接在紋理空間中進行學習的方法，尤其是在大規模數據集上的應用。本研究有別於傳統依賴預訓練二維擴散模型進行即時三維紋理優化的思路，轉而聚焦於UV紋理空間內在的學習機制這一基礎問題。我們首次訓練出能夠以前饋方式直接生成高解析度紋理貼圖的大規模擴散模型。為實現高解析度UV空間的高效學習，我們提出一種可擴展的網絡架構，該架構交替執行UV貼圖上的卷積運算與點雲上的注意力層計算。基於此架構設計，我們訓練出參數量達7億的擴散模型，可根據文字提示或單視角圖像生成UV紋理貼圖。訓練完成後的模型自然支援多種擴展應用，包括文字引導的紋理修補、稀疏視角紋理補全及文字驅動的紋理合成。項目頁面請見http://cvmi-lab.github.io/TEXGen/。

English

While high-quality texture maps are essential for realistic 3D asset rendering, few studies have explored learning directly in the texture space, especially on large-scale datasets. In this work, we depart from the conventional approach of relying on pre-trained 2D diffusion models for test-time optimization of 3D textures. Instead, we focus on the fundamental problem of learning in the UV texture space itself. For the first time, we train a large diffusion model capable of directly generating high-resolution texture maps in a feed-forward manner. To facilitate efficient learning in high-resolution UV spaces, we propose a scalable network architecture that interleaves convolutions on UV maps with attention layers on point clouds. Leveraging this architectural design, we train a 700 million parameter diffusion model that can generate UV texture maps guided by text prompts and single-view images. Once trained, our model naturally supports various extended applications, including text-guided texture inpainting, sparse-view texture completion, and text-driven texture synthesis. Project page is at http://cvmi-lab.github.io/TEXGen/.

TEXGen：網格紋理生成式擴散模型

TEXGen: a Generative Diffusion Model for Mesh Textures

摘要

Support