ControlMat：一种用于材质捕捉的受控生成方法

摘要

从照片中进行材料重建是3D内容创作民主化的关键组成部分。我们提议将这个不适定问题制定为受控合成问题，利用生成式深度网络的最新进展。我们提出了ControlMat方法，通过给定一张带有不受控照明的单张照片作为输入，将扩散模型调整为生成可信、可平铺、高分辨率的基于物理的数字材料。我们仔细分析了多通道输出的扩散模型的行为，调整采样过程以融合多尺度信息，并引入了滚动扩散，以实现平铺性和高分辨率输出的补丁扩散。我们的生成方法进一步允许探索各种可能对应于输入图像的材料，减轻未知照明条件的影响。我们展示了我们的方法优于最近的推断和潜空间优化方法，并仔细验证了我们的扩散过程设计选择。补充材料和额外细节可在以下网址获取：https://gvecchio.com/controlmat/。

English

Material reconstruction from a photograph is a key component of 3D content creation democratization. We propose to formulate this ill-posed problem as a controlled synthesis one, leveraging the recent progress in generative deep networks. We present ControlMat, a method which, given a single photograph with uncontrolled illumination as input, conditions a diffusion model to generate plausible, tileable, high-resolution physically-based digital materials. We carefully analyze the behavior of diffusion models for multi-channel outputs, adapt the sampling process to fuse multi-scale information and introduce rolled diffusion to enable both tileability and patched diffusion for high-resolution outputs. Our generative approach further permits exploration of a variety of materials which could correspond to the input image, mitigating the unknown lighting conditions. We show that our approach outperforms recent inference and latent-space-optimization methods, and carefully validate our diffusion process design choices. Supplemental materials and additional details are available at: https://gvecchio.com/controlmat/.

ControlMat：一种用于材质捕捉的受控生成方法

ControlMat: A Controlled Generative Approach to Material Capture

摘要

Support