FlashFace：具有高保真身份保存的人脸图像个性化

FlashFace: Human Image Personalization with High-fidelity Identity Preservation

March 25, 2024

作者: Shilong Zhang, Lianghua Huang, Xi Chen, Yifei Zhang, Zhi-Fan Wu, Yutong Feng, Wei Wang, Yujun Shen, Yu Liu, Ping Luo

cs.AI

摘要

本文介绍了FlashFace，这是一个实用工具，用户可以通过提供一个或几个参考面部图像和文本提示，即时轻松地个性化自己的照片。我们的方法与现有的人类照片定制方法有所区别，具有更高保真度的身份保留和更好的指导遵循，这得益于两个微妙的设计。首先，我们将面部身份编码为一系列特征图，而不是像以往那样一个图像标记，这使模型能够保留参考面部的更多细节（如疤痕、纹身和面部形状）。其次，我们引入了一种解耦集成策略，在文本到图像生成过程中平衡文本和图像引导，缓解参考面部和文本提示之间的冲突（例如，将成年人个性化为“儿童”或“老年人”）。大量实验结果证明了我们的方法在各种应用中的有效性，包括人类图像个性化、根据语言提示进行面部交换，将虚拟角色变成真实人物等。项目页面：https://jshilong.github.io/flashface-page。

English

This work presents FlashFace, a practical tool with which users can easily personalize their own photos on the fly by providing one or a few reference face images and a text prompt. Our approach is distinguishable from existing human photo customization methods by higher-fidelity identity preservation and better instruction following, benefiting from two subtle designs. First, we encode the face identity into a series of feature maps instead of one image token as in prior arts, allowing the model to retain more details of the reference faces (e.g., scars, tattoos, and face shape ). Second, we introduce a disentangled integration strategy to balance the text and image guidance during the text-to-image generation process, alleviating the conflict between the reference faces and the text prompts (e.g., personalizing an adult into a "child" or an "elder"). Extensive experimental results demonstrate the effectiveness of our method on various applications, including human image personalization, face swapping under language prompts, making virtual characters into real people, etc. Project Page: https://jshilong.github.io/flashface-page.