在擴散模型中測量風格相似性

摘要

生成模型現在被許多平面設計師和藝術家廣泛使用。先前的研究表明，這些模型在生成過程中記憶並經常複製訓練數據中的內容。因此，隨著它們的普及增加，逐次在將生成的圖像用於專業用途之前，進行數據庫搜索以確定圖像的特性是否可歸因於特定的訓練數據變得至關重要。目前用於此目的的現有工具專注於檢索具有相似語義內容的圖像。與此同時，許多藝術家關注文本到圖像模型中的風格複製。我們提出了一個框架，用於理解並從圖像中提取風格描述符。我們的框架包括一個新的數據集，根據一項洞察力精心策劃，即風格是圖像的一個主觀特性，捕捉到包括但不限於顏色、紋理、形狀等因素的複雜而有意義的交互作用。我們還提出了一種方法，用於提取風格描述符，這些描述符可用於將生成圖像的風格歸因於文本到圖像模型的訓練數據集中使用的圖像。我們展示了在各種風格檢索任務中的有希望的結果。我們還對Stable Diffusion模型中的風格歸因和匹配進行了定量和定性分析。代碼和工件可在https://github.com/learn2phoenix/CSD找到。

English

Generative models are now widely used by graphic designers and artists. Prior works have shown that these models remember and often replicate content from their training data during generation. Hence as their proliferation increases, it has become important to perform a database search to determine whether the properties of the image are attributable to specific training data, every time before a generated image is used for professional purposes. Existing tools for this purpose focus on retrieving images of similar semantic content. Meanwhile, many artists are concerned with style replication in text-to-image models. We present a framework for understanding and extracting style descriptors from images. Our framework comprises a new dataset curated using the insight that style is a subjective property of an image that captures complex yet meaningful interactions of factors including but not limited to colors, textures, shapes, etc. We also propose a method to extract style descriptors that can be used to attribute style of a generated image to the images used in the training dataset of a text-to-image model. We showcase promising results in various style retrieval tasks. We also quantitatively and qualitatively analyze style attribution and matching in the Stable Diffusion model. Code and artifacts are available at https://github.com/learn2phoenix/CSD.

在擴散模型中測量風格相似性

Measuring Style Similarity in Diffusion Models

摘要

Summary

Support

Support