TextureDreamer：通过几何感知扩散进行图像引导纹理合成

摘要

我们提出了TextureDreamer，这是一种新颖的图像引导纹理合成方法，可以将可重光纹理从少量输入图像（3至5张）转移到跨越任意类别的目标3D形状。纹理创建是视觉和图形领域的一个关键挑战。工业公司雇佣经验丰富的艺术家为3D资产手工制作纹理。传统方法需要密集采样的视图和精确对齐的几何形状，而基于学习的方法局限于数据集中特定类别的形状。相比之下，TextureDreamer可以从现实世界环境中将高度详细、复杂的纹理转移到任意对象，仅需几张随意拍摄的图像，潜在地极大地民主化了纹理创建。我们的核心思想，即个性化几何感知分数蒸馏（PGSD），汲取了最近扩散模型方面的进展，包括用于纹理信息提取的个性化建模、用于详细外观合成的变分分数蒸馏，以及带有ControlNet的显式几何指导。我们的整合和几个重要修改显著提高了纹理质量。对跨越不同类别的真实图像进行的实验表明，TextureDreamer能够成功地将高度逼真、语义丰富的纹理转移到任意对象，超越了先前最先进技术的视觉质量。

English

We present TextureDreamer, a novel image-guided texture synthesis method to transfer relightable textures from a small number of input images (3 to 5) to target 3D shapes across arbitrary categories. Texture creation is a pivotal challenge in vision and graphics. Industrial companies hire experienced artists to manually craft textures for 3D assets. Classical methods require densely sampled views and accurately aligned geometry, while learning-based methods are confined to category-specific shapes within the dataset. In contrast, TextureDreamer can transfer highly detailed, intricate textures from real-world environments to arbitrary objects with only a few casually captured images, potentially significantly democratizing texture creation. Our core idea, personalized geometry-aware score distillation (PGSD), draws inspiration from recent advancements in diffuse models, including personalized modeling for texture information extraction, variational score distillation for detailed appearance synthesis, and explicit geometry guidance with ControlNet. Our integration and several essential modifications substantially improve the texture quality. Experiments on real images spanning different categories show that TextureDreamer can successfully transfer highly realistic, semantic meaningful texture to arbitrary objects, surpassing the visual quality of previous state-of-the-art.

TextureDreamer：通过几何感知扩散进行图像引导纹理合成

TextureDreamer: Image-guided Texture Synthesis through Geometry-aware Diffusion

摘要

Support