文字到圖像個人化模型中的潛在身份微調

摘要

生成和编辑人脸需要极高的精度，因为即便是细微的修改也可能显著改变被识别对象的身份特征。然而，当前基于通用文本到图像模型构建的个性化与编辑方法，往往缺乏实现精细面部编辑所需的精度。我们提出一种用于文本到图像个性化模型的细粒度身份调优方法。与标准图像编辑（操作给定图像）不同，身份调优会修改特定身份的潜在表示，从而生成持续呈现同一编辑后身份的多样化图像。为实现细粒度的潜在身份调优，我们探索了用于文本到图像个性化预训练冻结编码器的潜在空间。该方法无需额外训练，而是利用冻结编码器现有架构揭示潜在语义方向。该空间由一组潜在标记构成，这些标记在捕捉身份不同方面中扮演独特角色，通常对应特定空间或语义面部区域。我们证明，在此空间及由选定标记定义的子空间中可识别有意义的方向，从而实现局域化、细粒度且语义一致的编辑。通过定性与定量实验验证了该方法在实现多样化局域面部编辑的同时保持跨图像身份一致性。项目页面：https://garibida.github.io/IdentityTuning/

English

Generating and editing a person's face demands high precision, as even minor modifications can significantly alter a subject's perceived identity. Current personalization and editing methods built on general-purpose text-to-image models, however, often lack the precision required for fine-grained facial edits. We present a method for fine-grained identity tuning in text-to-image personalization models. Unlike standard image editing, which operates on a given image, identity tuning modifies the latent representation of a specific identity, enabling the generation of diverse images that consistently depict the same edited identity. To enable fine-grained latent identity tuning, we explore the latent space of a pre-trained, frozen encoder for text-to-image personalization. Our approach requires no additional training. Instead, it leverages the existing architecture of a frozen encoder to uncover latent semantic directions. This space consists of a set of latent tokens that play distinct roles in capturing different aspects of an identity and often correspond to specific spatial or semantic facial regions. We show that meaningful directions can be identified within this space and within subspaces defined by selected tokens, enabling localized, fine-grained, and semantically coherent edits. We validate our approach through qualitative and quantitative experiments that demonstrate diverse localized facial edits while preserving cross-image identity consistency. Project page at: https://garibida.github.io/IdentityTuning/