MARBLE：CLIP空间中的材质重组与融合

摘要

基于示例图像对物体材质进行编辑是计算机视觉与图形学领域的一个活跃研究方向。我们提出了MARBLE方法，通过在CLIP空间中寻找材质嵌入并利用其控制预训练的文本到图像模型，实现材质混合与细粒度材质属性的重组。我们通过定位去噪UNet中负责材质归因的模块，改进了基于示例的材质编辑。给定两幅材质示例图像，我们在CLIP空间中寻找混合材质的方向。此外，借助浅层网络预测期望材质属性变化的方向，我们能够对粗糙度、金属感、透明度及发光等细粒度材质属性实现参数化控制。我们通过定性与定量分析，验证了所提方法的有效性。同时，展示了该方法在单次前向传播中执行多重编辑的能力及其在绘画领域的适用性。项目页面：https://marblecontrol.github.io/

English

Editing materials of objects in images based on exemplar images is an active area of research in computer vision and graphics. We propose MARBLE, a method for performing material blending and recomposing fine-grained material properties by finding material embeddings in CLIP-space and using that to control pre-trained text-to-image models. We improve exemplar-based material editing by finding a block in the denoising UNet responsible for material attribution. Given two material exemplar-images, we find directions in the CLIP-space for blending the materials. Further, we can achieve parametric control over fine-grained material attributes such as roughness, metallic, transparency, and glow using a shallow network to predict the direction for the desired material attribute change. We perform qualitative and quantitative analysis to demonstrate the efficacy of our proposed method. We also present the ability of our method to perform multiple edits in a single forward pass and applicability to painting. Project Page: https://marblecontrol.github.io/

MARBLE：CLIP空间中的材质重组与融合

MARBLE: Material Recomposition and Blending in CLIP-Space

摘要

Support