藝術家：無需訓練的美學可控文字驅動風格化

Artist: Aesthetically Controllable Text-Driven Stylization without Training

July 22, 2024

作者: Ruixiang Jiang, Changwen Chen

cs.AI

摘要

擴散模型在去噪過程中紡織內容和風格生成，直接應用於風格化任務時可能導致不希望的內容修改。現有方法難以有效控制擴散模型以滿足風格化的審美要求。本文介紹一種名為「Artist」的訓練免費方法，用於美學控制預訓練擴散模型的內容和風格生成，以進行以文本驅動的風格化。我們的關鍵見解是將內容和風格的去噪分為單獨的擴散過程，同時在它們之間共享信息。我們提出了簡單而有效的內容和風格控制方法，抑制與風格無關的內容生成，從而產生和諧的風格化結果。大量實驗表明，我們的方法在實現審美級風格化要求方面表現優異，保留了內容圖像中的細節並與風格提示相匹配。此外，我們展示了從各種角度高度可控的風格化強度。代碼將被釋出，項目主頁：https://DiffusionArtist.github.io

English

Diffusion models entangle content and style generation during the denoising process, leading to undesired content modification when directly applied to stylization tasks. Existing methods struggle to effectively control the diffusion model to meet the aesthetic-level requirements for stylization. In this paper, we introduce Artist, a training-free approach that aesthetically controls the content and style generation of a pretrained diffusion model for text-driven stylization. Our key insight is to disentangle the denoising of content and style into separate diffusion processes while sharing information between them. We propose simple yet effective content and style control methods that suppress style-irrelevant content generation, resulting in harmonious stylization results. Extensive experiments demonstrate that our method excels at achieving aesthetic-level stylization requirements, preserving intricate details in the content image and aligning well with the style prompt. Furthermore, we showcase the highly controllability of the stylization strength from various perspectives. Code will be released, project home page: https://DiffusionArtist.github.io