3DTopia-XL：通过基元扩散实现高质量3D资产生成规模化

3DTopia-XL: Scaling High-quality 3D Asset Generation via Primitive Diffusion

September 19, 2024

作者: Zhaoxi Chen, Jiaxiang Tang, Yuhao Dong, Ziang Cao, Fangzhou Hong, Yushi Lan, Tengfei Wang, Haozhe Xie, Tong Wu, Shunsuke Saito, Liang Pan, Dahua Lin, Ziwei Liu

cs.AI

摘要

随着各行业对高质量三维资产的需求不断增加，需要高效自动化的三维内容创作。尽管三维生成模型近年来取得了进展，但现有方法仍面临优化速度、几何保真度以及缺乏适用于基于物理的渲染（PBR）的资产等挑战。本文介绍了3DTopia-XL，一种可扩展的本机三维生成模型，旨在克服这些限制。3DTopia-XL利用一种新颖的基于基元的三维表示，PrimX，将详细形状、反照率和材质字段编码为紧凑的张量格式，有助于使用PBR资产建模高分辨率几何。在这种新颖表示的基础上，我们提出了一个基于扩散变换器（DiT）的生成框架，包括1）基元补丁压缩，2）和潜在基元扩散。3DTopia-XL学习从文本或视觉输入生成高质量三维资产。我们进行了广泛的定性和定量实验，证明了3DTopia-XL在生成具有细粒度纹理和材质的高质量三维资产方面明显优于现有方法，有效地弥合了生成模型与实际应用之间的质量差距。

English

The increasing demand for high-quality 3D assets across various industries necessitates efficient and automated 3D content creation. Despite recent advancements in 3D generative models, existing methods still face challenges with optimization speed, geometric fidelity, and the lack of assets for physically based rendering (PBR). In this paper, we introduce 3DTopia-XL, a scalable native 3D generative model designed to overcome these limitations. 3DTopia-XL leverages a novel primitive-based 3D representation, PrimX, which encodes detailed shape, albedo, and material field into a compact tensorial format, facilitating the modeling of high-resolution geometry with PBR assets. On top of the novel representation, we propose a generative framework based on Diffusion Transformer (DiT), which comprises 1) Primitive Patch Compression, 2) and Latent Primitive Diffusion. 3DTopia-XL learns to generate high-quality 3D assets from textual or visual inputs. We conduct extensive qualitative and quantitative experiments to demonstrate that 3DTopia-XL significantly outperforms existing methods in generating high-quality 3D assets with fine-grained textures and materials, efficiently bridging the quality gap between generative models and real-world applications.