InstantID：零-shot 身份保留生成，秒速完成

摘要

个性化图像合成领域取得了显著进展，如文本反演、梦境展示和LoRA等方法。然而，它们在实际应用中受到高存储需求、漫长的微调过程以及需要多个参考图像的限制。相反，现有的基于ID嵌入的方法虽然只需要单向推理，但面临挑战：它们要么需要跨多个模型参数进行大量微调，要么与社区预训练模型不兼容，要么无法保持高面部保真度。为了解决这些限制，我们引入了InstantID，这是一个基于强大扩散模型的解决方案。我们的即插即用模块能够灵活处理各种风格的图像个性化，只需一张面部图像，同时确保高保真度。为实现这一目标，我们设计了一个新颖的IdentityNet，通过施加强语义和弱空间条件，将面部和标志图像与文本提示相结合，引导图像生成。InstantID展示了出色的性能和效率，在重视身份保护的实际应用中具有极大的益处。此外，我们的工作与流行的预训练文本到图像扩散模型（如SD1.5和SDXL）无缝集成，作为一个适应性插件。我们的代码和预训练检查点将在https://github.com/InstantID/InstantID 上提供。

English

There has been significant progress in personalized image synthesis with methods such as Textual Inversion, DreamBooth, and LoRA. Yet, their real-world applicability is hindered by high storage demands, lengthy fine-tuning processes, and the need for multiple reference images. Conversely, existing ID embedding-based methods, while requiring only a single forward inference, face challenges: they either necessitate extensive fine-tuning across numerous model parameters, lack compatibility with community pre-trained models, or fail to maintain high face fidelity. Addressing these limitations, we introduce InstantID, a powerful diffusion model-based solution. Our plug-and-play module adeptly handles image personalization in various styles using just a single facial image, while ensuring high fidelity. To achieve this, we design a novel IdentityNet by imposing strong semantic and weak spatial conditions, integrating facial and landmark images with textual prompts to steer the image generation. InstantID demonstrates exceptional performance and efficiency, proving highly beneficial in real-world applications where identity preservation is paramount. Moreover, our work seamlessly integrates with popular pre-trained text-to-image diffusion models like SD1.5 and SDXL, serving as an adaptable plugin. Our codes and pre-trained checkpoints will be available at https://github.com/InstantID/InstantID.

InstantID：零-shot 身份保留生成，秒速完成

InstantID: Zero-shot Identity-Preserving Generation in Seconds

摘要

Support