InstantStyle-Plus：在文本到图像生成中实现风格转移并保留内容

InstantStyle-Plus: Style Transfer with Content-Preserving in Text-to-Image Generation

June 30, 2024

作者: Haofan Wang, Peng Xing, Renyuan Huang, Hao Ai, Qixun Wang, Xu Bai

cs.AI

摘要

风格迁移是一种创新的过程，旨在创造一幅保留原始本质但融合另一种视觉风格的图像。尽管扩散模型在个性化主题驱动或风格驱动应用中展示出令人印象深刻的生成能力，但现有的最先进方法仍然在实现内容保留和风格增强之间的无缝平衡方面遇到困难。例如，放大风格的影响往往会削弱内容的结构完整性。为了解决这些挑战，我们将风格迁移任务分解为三个核心要素：1) 风格，专注于图像的美学特征；2) 空间结构，涉及视觉元素的几何排列和构图；以及3) 语义内容，捕捉图像的概念含义。在这些原则的指导下，我们引入了InstantStyle-Plus，这是一种强调保持原始内容完整性并无缝集成目标风格的方法。具体而言，我们的方法通过一种高效、轻量级的过程实现风格注入，利用了尖端的InstantStyle框架。为了加强内容保留，我们首先使用反转的内容潜在噪声和一个多功能的即插即用的Tile ControlNet来保留原始图像的固有布局。我们还结合了全局语义适配器来增强语义内容的保真度。为了防止风格信息的稀释，我们使用风格提取器作为鉴别器，提供补充的风格指导。代码将在https://github.com/instantX-research/InstantStyle-Plus 上提供。

English

Style transfer is an inventive process designed to create an image that maintains the essence of the original while embracing the visual style of another. Although diffusion models have demonstrated impressive generative power in personalized subject-driven or style-driven applications, existing state-of-the-art methods still encounter difficulties in achieving a seamless balance between content preservation and style enhancement. For example, amplifying the style's influence can often undermine the structural integrity of the content. To address these challenges, we deconstruct the style transfer task into three core elements: 1) Style, focusing on the image's aesthetic characteristics; 2) Spatial Structure, concerning the geometric arrangement and composition of visual elements; and 3) Semantic Content, which captures the conceptual meaning of the image. Guided by these principles, we introduce InstantStyle-Plus, an approach that prioritizes the integrity of the original content while seamlessly integrating the target style. Specifically, our method accomplishes style injection through an efficient, lightweight process, utilizing the cutting-edge InstantStyle framework. To reinforce the content preservation, we initiate the process with an inverted content latent noise and a versatile plug-and-play tile ControlNet for preserving the original image's intrinsic layout. We also incorporate a global semantic adapter to enhance the semantic content's fidelity. To safeguard against the dilution of style information, a style extractor is employed as discriminator for providing supplementary style guidance. Codes will be available at https://github.com/instantX-research/InstantStyle-Plus.