NaTex：作为潜在颜色扩散的无缝纹理生成

摘要

我们提出NaTex——一种直接在三维空间中预测纹理颜色的原生纹理生成框架。与以往依赖烘焙由几何条件多视图扩散模型（MVDs）生成的二维多视角图像的方法不同，NaTex规避了MVD流程的若干固有局限。这些局限包括：处理需修复的遮挡区域困难、难以实现边界处网格与纹理的精准对齐、以及保持跨视角内容与色彩强度的一致性和连贯性。NaTex采用创新范式，将纹理视作密集彩色点云，从而解决上述问题。基于这一理念，我们提出潜空间色彩扩散技术，包含几何感知的彩色点云VAE和多控制扩散Transformer（DiT）——整套系统使用三维数据从头训练，专用于纹理重建与生成。为实现精准对齐，我们引入原生几何控制机制，通过位置编码和几何潜变量将直接三维空间信息作为DiT的条件输入。我们协同设计了VAE-DiT架构：几何潜变量由与色彩VAE紧密耦合的专用几何分支提取，提供与纹理保持强对应关系的细粒度表面引导。通过这些设计，NaTex展现出强大性能，在纹理连贯性与对齐精度上显著超越现有方法。此外，NaTex还表现出优异的泛化能力，无需训练或仅需简单调参即可适用于材质生成、纹理优化、部件分割与纹理映射等多种下游任务。

English

We present NaTex, a native texture generation framework that predicts texture color directly in 3D space. In contrast to previous approaches that rely on baking 2D multi-view images synthesized by geometry-conditioned Multi-View Diffusion models (MVDs), NaTex avoids several inherent limitations of the MVD pipeline. These include difficulties in handling occluded regions that require inpainting, achieving precise mesh-texture alignment along boundaries, and maintaining cross-view consistency and coherence in both content and color intensity. NaTex features a novel paradigm that addresses the aforementioned issues by viewing texture as a dense color point cloud. Driven by this idea, we propose latent color diffusion, which comprises a geometry-awared color point cloud VAE and a multi-control diffusion transformer (DiT), entirely trained from scratch using 3D data, for texture reconstruction and generation. To enable precise alignment, we introduce native geometry control that conditions the DiT on direct 3D spatial information via positional embeddings and geometry latents. We co-design the VAE-DiT architecture, where the geometry latents are extracted via a dedicated geometry branch tightly coupled with the color VAE, providing fine-grained surface guidance that maintains strong correspondence with the texture. With these designs, NaTex demonstrates strong performance, significantly outperforming previous methods in texture coherence and alignment. Moreover, NaTex also exhibits strong generalization capabilities, either training-free or with simple tuning, for various downstream applications, e.g., material generation, texture refinement, and part segmentation and texturing.

NaTex：作为潜在颜色扩散的无缝纹理生成

NaTex: Seamless Texture Generation as Latent Color Diffusion

摘要

Support