CreativeSynth：基於多模擬擴散的視覺藝術創意融合與合成

摘要

大規模文本到圖像生成模型取得了顯著進展，展示了它們合成各種高質量圖像的能力。然而，將這些模型適應到藝術圖像編輯中存在兩個重要挑戰。首先，用戶難以精心製作詳細描述輸入圖像視覺元素的文本提示。其次，當主流模型在特定區域進行修改時，經常會破壞整體藝術風格，使得實現連貫和美學統一的藝術作品變得複雜。為了克服這些障礙，我們建立了創新的統一框架CreativeSynth，它基於擴散模型，具有協調多模態輸入和在藝術圖像生成領域中多任務的能力。通過將多模態特徵與定制的注意機制相結合，CreativeSynth 促進了將現實世界語義內容透過反演和實時風格轉移輸入到藝術領域。這允許對圖像風格和內容進行精確操作，同時保持原始模型參數的完整性。嚴格的定性和定量評估突顯了CreativeSynth 在增強藝術圖像的保真度方面的優越性，並保留了其固有的美學本質。通過彌合生成模型和藝術精髓之間的差距，CreativeSynth 成為了一個定制的數字調色板。

English

Large-scale text-to-image generative models have made impressive strides, showcasing their ability to synthesize a vast array of high-quality images. However, adapting these models for artistic image editing presents two significant challenges. Firstly, users struggle to craft textual prompts that meticulously detail visual elements of the input image. Secondly, prevalent models, when effecting modifications in specific zones, frequently disrupt the overall artistic style, complicating the attainment of cohesive and aesthetically unified artworks. To surmount these obstacles, we build the innovative unified framework CreativeSynth, which is based on a diffusion model with the ability to coordinate multimodal inputs and multitask in the field of artistic image generation. By integrating multimodal features with customized attention mechanisms, CreativeSynth facilitates the importation of real-world semantic content into the domain of art through inversion and real-time style transfer. This allows for the precise manipulation of image style and content while maintaining the integrity of the original model parameters. Rigorous qualitative and quantitative evaluations underscore that CreativeSynth excels in enhancing artistic images' fidelity and preserves their innate aesthetic essence. By bridging the gap between generative models and artistic finesse, CreativeSynth becomes a custom digital palette.

CreativeSynth：基於多模擬擴散的視覺藝術創意融合與合成

CreativeSynth: Creative Blending and Synthesis of Visual Arts based on Multimodal Diffusion

摘要

Support