TextureDreamer：通過幾何感知擴散進行圖像導向紋理合成

摘要

我們提出了TextureDreamer，一種新穎的圖像導向紋理合成方法，可將可重新照明的紋理從少量輸入圖像（3至5張）轉移到跨越任意類別的目標3D形狀。紋理創建是視覺和圖形領域的一個關鍵挑戰。工業公司聘請經驗豐富的藝術家手工製作3D資產的紋理。傳統方法需要密集採樣的視圖和準確對齊的幾何形狀，而基於學習的方法則僅限於數據集中特定類別的形狀。相比之下，TextureDreamer可以從現實環境中僅通過幾張隨意拍攝的圖像將高度詳細、複雜的紋理轉移到任意物體，潛在地顯著民主化紋理創建。我們的核心思想，個性化幾何感知分數提煉（PGSD），受到最近擴散模型方面的進展的啟發，包括用於紋理信息提取的個性化建模、用於詳細外觀合成的變分分數提煉，以及具有ControlNet的明確幾何引導。我們的整合和幾個重要修改顯著改善了紋理質量。對跨越不同類別的真實圖像進行的實驗表明，TextureDreamer可以成功地將高度逼真、語義有意義的紋理轉移到任意物體，超越了先前最先進技術的視覺質量。

English

We present TextureDreamer, a novel image-guided texture synthesis method to transfer relightable textures from a small number of input images (3 to 5) to target 3D shapes across arbitrary categories. Texture creation is a pivotal challenge in vision and graphics. Industrial companies hire experienced artists to manually craft textures for 3D assets. Classical methods require densely sampled views and accurately aligned geometry, while learning-based methods are confined to category-specific shapes within the dataset. In contrast, TextureDreamer can transfer highly detailed, intricate textures from real-world environments to arbitrary objects with only a few casually captured images, potentially significantly democratizing texture creation. Our core idea, personalized geometry-aware score distillation (PGSD), draws inspiration from recent advancements in diffuse models, including personalized modeling for texture information extraction, variational score distillation for detailed appearance synthesis, and explicit geometry guidance with ControlNet. Our integration and several essential modifications substantially improve the texture quality. Experiments on real images spanning different categories show that TextureDreamer can successfully transfer highly realistic, semantic meaningful texture to arbitrary objects, surpassing the visual quality of previous state-of-the-art.

TextureDreamer：通過幾何感知擴散進行圖像導向紋理合成

TextureDreamer: Image-guided Texture Synthesis through Geometry-aware Diffusion

摘要

Support