ControlMat：一種受控生成材料捕獲方法

摘要

從照片中重建材料是3D內容創建民主化的關鍵組成部分。我們提議將這個不透明的問題定義為一個受控合成問題，利用生成式深度網絡的最新進展。我們提出了ControlMat方法，通過給定一張具有不受控制照明的照片作為輸入，條件一個擴散模型以生成可信的、可平鋪的、高分辨率的基於物理的數字材料。我們仔細分析了多通道輸出的擴散模型的行為，調整取樣過程以融合多尺度信息，並引入滾動擴散，以實現平鋪性和高分辨率輸出的補丁擴散。我們的生成方法進一步允許探索各種可能對應於輸入圖像的材料，減輕未知照明條件的影響。我們展示了我們的方法優於最近的推斷和潛在空間優化方法，並仔細驗證了我們的擴散過程設計選擇。補充材料和更多細節可在以下網址獲得：https://gvecchio.com/controlmat/。

English

Material reconstruction from a photograph is a key component of 3D content creation democratization. We propose to formulate this ill-posed problem as a controlled synthesis one, leveraging the recent progress in generative deep networks. We present ControlMat, a method which, given a single photograph with uncontrolled illumination as input, conditions a diffusion model to generate plausible, tileable, high-resolution physically-based digital materials. We carefully analyze the behavior of diffusion models for multi-channel outputs, adapt the sampling process to fuse multi-scale information and introduce rolled diffusion to enable both tileability and patched diffusion for high-resolution outputs. Our generative approach further permits exploration of a variety of materials which could correspond to the input image, mitigating the unknown lighting conditions. We show that our approach outperforms recent inference and latent-space-optimization methods, and carefully validate our diffusion process design choices. Supplemental materials and additional details are available at: https://gvecchio.com/controlmat/.

ControlMat：一種受控生成材料捕獲方法

ControlMat: A Controlled Generative Approach to Material Capture

摘要

Support