评估扩散模型中的风格相似度

摘要

生成模型如今被广泛应用于图形设计师和艺术家的创作中。先前的研究已表明，这些模型在生成过程中会记忆并经常复制其训练数据中的内容。因此，随着这些模型的普及度增加，每次在专业用途上使用生成图像之前，进行数据库搜索以确定图像属性是否源自特定训练数据变得尤为重要。现有的工具主要集中于检索语义内容相似的图像。与此同时，许多艺术家对文本到图像模型中的风格复制问题表示关切。我们提出了一种理解和提取图像风格描述符的框架。该框架包含一个新数据集，该数据集的构建基于以下见解：风格是图像的主观属性，它捕捉了包括但不限于颜色、纹理、形状等因素之间的复杂而有意义的交互。我们还提出了一种方法，用于提取风格描述符，这些描述符可用于将生成图像的风格归因于文本到图像模型的训练数据集中的图像。我们在多种风格检索任务中展示了有前景的结果。此外，我们还对Stable Diffusion模型中的风格归因和匹配进行了定量和定性的分析。代码和相关资源可在https://github.com/learn2phoenix/CSD获取。

English

Generative models are now widely used by graphic designers and artists. Prior works have shown that these models remember and often replicate content from their training data during generation. Hence as their proliferation increases, it has become important to perform a database search to determine whether the properties of the image are attributable to specific training data, every time before a generated image is used for professional purposes. Existing tools for this purpose focus on retrieving images of similar semantic content. Meanwhile, many artists are concerned with style replication in text-to-image models. We present a framework for understanding and extracting style descriptors from images. Our framework comprises a new dataset curated using the insight that style is a subjective property of an image that captures complex yet meaningful interactions of factors including but not limited to colors, textures, shapes, etc. We also propose a method to extract style descriptors that can be used to attribute style of a generated image to the images used in the training dataset of a text-to-image model. We showcase promising results in various style retrieval tasks. We also quantitatively and qualitatively analyze style attribution and matching in the Stable Diffusion model. Code and artifacts are available at https://github.com/learn2phoenix/CSD.

评估扩散模型中的风格相似度

Measuring Style Similarity in Diffusion Models

摘要

Summary

Support

Support