FlashFace：具有高保真身份保存的人像個性化

FlashFace: Human Image Personalization with High-fidelity Identity Preservation

March 25, 2024

作者: Shilong Zhang, Lianghua Huang, Xi Chen, Yifei Zhang, Zhi-Fan Wu, Yutong Feng, Wei Wang, Yujun Shen, Yu Liu, Ping Luo

cs.AI

摘要

本研究提出了FlashFace，一個實用工具，讓使用者可以透過提供一個或幾個參考臉部圖像和文字提示來即時個性化自己的照片。我們的方法與現有的人類照片定制方法有所不同，具有更高保留身份特徵的保真度和更好的指導遵循，這得益於兩個微妙的設計。首先，我們將臉部身份編碼為一系列特徵圖，而不是像以往的方法中的單一圖像標記，這使模型能夠保留參考臉部的更多細節（例如疤痕、紋身和臉部形狀）。其次，我們引入了一種解耦合的整合策略，在文本轉圖像生成過程中平衡文本和圖像引導，緩解參考臉部與文本提示之間的衝突（例如，將成年人個性化為"孩子"或"老人"）。大量實驗結果證明了我們的方法在各種應用中的有效性，包括人類圖像個性化、根據語言提示進行臉部交換、將虛擬角色變成真實人物等。項目頁面：https://jshilong.github.io/flashface-page。

English

This work presents FlashFace, a practical tool with which users can easily personalize their own photos on the fly by providing one or a few reference face images and a text prompt. Our approach is distinguishable from existing human photo customization methods by higher-fidelity identity preservation and better instruction following, benefiting from two subtle designs. First, we encode the face identity into a series of feature maps instead of one image token as in prior arts, allowing the model to retain more details of the reference faces (e.g., scars, tattoos, and face shape ). Second, we introduce a disentangled integration strategy to balance the text and image guidance during the text-to-image generation process, alleviating the conflict between the reference faces and the text prompts (e.g., personalizing an adult into a "child" or an "elder"). Extensive experimental results demonstrate the effectiveness of our method on various applications, including human image personalization, face swapping under language prompts, making virtual characters into real people, etc. Project Page: https://jshilong.github.io/flashface-page.