IDAdapter：学习混合特征，实现无需调整的个性化文本到图像模型

IDAdapter: Learning Mixed Features for Tuning-Free Personalization of Text-to-Image Models

March 20, 2024

作者: Siying Cui, Jiankang Deng, Jia Guo, Xiang An, Yongle Zhao, Xinyu Wei, Ziyong Feng

cs.AI

摘要

利用稳定扩散技术生成个性化肖像已成为一种强大且显著的工具，使用户能够基于特定提示创建高保真度的定制角色头像。然而，现有的个性化方法面临挑战，包括测试时微调、需要多个输入图像、身份保留较低以及生成结果多样性有限。为了克服这些挑战，我们引入了IDAdapter，这是一种无需微调的方法，可以增强从单张面部图像生成个性化图像时的多样性和身份保留。IDAdapter通过将个性化概念融入生成过程中，结合文本和视觉注入以及面部身份损失。在训练阶段，我们将特定身份的多个参考图像的混合特征纳入，以丰富与身份相关的内容细节，引导模型生成比以往作品更具多样化风格、表情和角度的图像。广泛的评估表明我们的方法的有效性，实现了生成图像中的多样性和身份保真度。

English

Leveraging Stable Diffusion for the generation of personalized portraits has emerged as a powerful and noteworthy tool, enabling users to create high-fidelity, custom character avatars based on their specific prompts. However, existing personalization methods face challenges, including test-time fine-tuning, the requirement of multiple input images, low preservation of identity, and limited diversity in generated outcomes. To overcome these challenges, we introduce IDAdapter, a tuning-free approach that enhances the diversity and identity preservation in personalized image generation from a single face image. IDAdapter integrates a personalized concept into the generation process through a combination of textual and visual injections and a face identity loss. During the training phase, we incorporate mixed features from multiple reference images of a specific identity to enrich identity-related content details, guiding the model to generate images with more diverse styles, expressions, and angles compared to previous works. Extensive evaluations demonstrate the effectiveness of our method, achieving both diversity and identity fidelity in generated images.