IDAdapter：學習混合特徵以實現無需調整的個性化文本到圖像模型

IDAdapter: Learning Mixed Features for Tuning-Free Personalization of Text-to-Image Models

March 20, 2024

作者: Siying Cui, Jiankang Deng, Jia Guo, Xiang An, Yongle Zhao, Xinyu Wei, Ziyong Feng

cs.AI

摘要

利用穩定擴散技術生成個性化肖像已成為一種強大且值得注意的工具，使用戶能夠根據特定提示創建高保真度的自定義角色頭像。然而，現有的個性化方法面臨著挑戰，包括測試時間微調、需要多個輸入圖像、身份保存率低以及生成結果的多樣性有限。為了克服這些挑戰，我們引入了IDAdapter，這是一種無需調整的方法，可從單張面部圖像中增強個性化圖像生成中的多樣性和身份保存率。IDAdapter通過文本和視覺注入以及面部身份損失的結合將個性化概念融入生成過程中。在訓練階段，我們從特定身份的多個參考圖像中綜合特徵，以豐富與身份相關的內容細節，引導模型生成比以往更具多樣風格、表情和角度的圖像。廣泛的評估證明了我們方法的有效性，在生成的圖像中實現了多樣性和身份保真度。

English

Leveraging Stable Diffusion for the generation of personalized portraits has emerged as a powerful and noteworthy tool, enabling users to create high-fidelity, custom character avatars based on their specific prompts. However, existing personalization methods face challenges, including test-time fine-tuning, the requirement of multiple input images, low preservation of identity, and limited diversity in generated outcomes. To overcome these challenges, we introduce IDAdapter, a tuning-free approach that enhances the diversity and identity preservation in personalized image generation from a single face image. IDAdapter integrates a personalized concept into the generation process through a combination of textual and visual injections and a face identity loss. During the training phase, we incorporate mixed features from multiple reference images of a specific identity to enrich identity-related content details, guiding the model to generate images with more diverse styles, expressions, and angles compared to previous works. Extensive evaluations demonstrate the effectiveness of our method, achieving both diversity and identity fidelity in generated images.