PhotoVerse：使用文本到图像扩散模型进行无需调整的图像定制

摘要

个性化文本到图像生成已经成为一种强大且备受追捧的工具，使用户能够基于其特定概念和提示创建定制图像。然而，现有的个性化方法面临诸多挑战，包括长时间调整、大量存储需求、每个身份需要多个输入图像以及在保留身份和可编辑性方面存在限制。为了解决这些障碍，我们提出了PhotoVerse，这是一种创新方法，它在文本和图像领域都融入了双分支调节机制，有效控制图像生成过程。此外，我们引入了面部身份损失作为一种新颖组件，以增强训练过程中对身份的保留。值得注意的是，我们提出的PhotoVerse 消除了测试时间调整的需要，仅依赖于目标身份的单张面部照片，大大降低了与图像生成相关的资源成本。经过单一训练阶段，我们的方法能够在几秒钟内生成高质量图像。此外，我们的方法可以生成包含各种场景和风格的多样化图像。广泛的评估证明了我们的方法的卓越性能，实现了保留身份和促进可编辑性的双重目标。项目页面：https://photoverse2d.github.io/

English

Personalized text-to-image generation has emerged as a powerful and sought-after tool, empowering users to create customized images based on their specific concepts and prompts. However, existing approaches to personalization encounter multiple challenges, including long tuning times, large storage requirements, the necessity for multiple input images per identity, and limitations in preserving identity and editability. To address these obstacles, we present PhotoVerse, an innovative methodology that incorporates a dual-branch conditioning mechanism in both text and image domains, providing effective control over the image generation process. Furthermore, we introduce facial identity loss as a novel component to enhance the preservation of identity during training. Remarkably, our proposed PhotoVerse eliminates the need for test time tuning and relies solely on a single facial photo of the target identity, significantly reducing the resource cost associated with image generation. After a single training phase, our approach enables generating high-quality images within only a few seconds. Moreover, our method can produce diverse images that encompass various scenes and styles. The extensive evaluation demonstrates the superior performance of our approach, which achieves the dual objectives of preserving identity and facilitating editability. Project page: https://photoverse2d.github.io/

PhotoVerse：使用文本到图像扩散模型进行无需调整的图像定制

PhotoVerse: Tuning-Free Image Customization with Text-to-Image Diffusion Models

摘要

Support