文本到图像个性化模型中的潜在身份调优

摘要

生成和编辑人脸需要极高的精度，因为即使是细微的改动也可能显著改变被识别对象的身份特征。然而，当前基于通用文本到图像模型的个性化与编辑方法，往往缺乏实现精细面部编辑所需的精度。我们提出了一种方法，用于对文本到图像个性化模型进行细粒度的身份微调。与标准图像编辑（对给定图像进行操作）不同，身份微调会修改特定身份的潜在表示，从而生成多种一致呈现同一编辑后身份的多样化图像。为了实现细粒度的潜在身份微调，我们探索了用于文本到图像个性化的预训练冻结编码器的潜在空间。我们的方法无需额外训练，而是利用冻结编码器的现有架构来揭示潜在语义方向。该空间由一组潜在标记组成，这些标记在捕捉身份的不同方面扮演着独特角色，并且通常对应于特定的空间或语义面部区域。我们证明了在该空间内以及由选定标记定义的子空间中，可以识别出有意义的语义方向，从而实现局部化、细粒度且语义连贯的编辑。通过定性和定量实验，我们验证了该方法能够在保持跨图像身份一致性的同时，实现多样化的局部面部编辑。项目页面：https://garibida.github.io/IdentityTuning/

English

Generating and editing a person's face demands high precision, as even minor modifications can significantly alter a subject's perceived identity. Current personalization and editing methods built on general-purpose text-to-image models, however, often lack the precision required for fine-grained facial edits. We present a method for fine-grained identity tuning in text-to-image personalization models. Unlike standard image editing, which operates on a given image, identity tuning modifies the latent representation of a specific identity, enabling the generation of diverse images that consistently depict the same edited identity. To enable fine-grained latent identity tuning, we explore the latent space of a pre-trained, frozen encoder for text-to-image personalization. Our approach requires no additional training. Instead, it leverages the existing architecture of a frozen encoder to uncover latent semantic directions. This space consists of a set of latent tokens that play distinct roles in capturing different aspects of an identity and often correspond to specific spatial or semantic facial regions. We show that meaningful directions can be identified within this space and within subspaces defined by selected tokens, enabling localized, fine-grained, and semantically coherent edits. We validate our approach through qualitative and quantitative experiments that demonstrate diverse localized facial edits while preserving cross-image identity consistency. Project page at: https://garibida.github.io/IdentityTuning/