EXPL-FR：通过视觉-语言对齐解释人脸识别模型

摘要

深度人脸识别（FR）模型已达到近乎饱和的准确率，但仍然是透明的反面——从业者无法询问相似度得分依赖了哪些语义属性。EXPL-FR 在 FR 模型自身的嵌入空间内回答了这一难题。一个轻量级适配器将视觉-语言模型（VLM）的图像编码器与冻结的 FR 空间对齐，该适配器仅在人脸图像上训练，从未使用文本。由于 VLM 的编码器共享同一空间，同一适配器也适用于文本编码器，从而将 22 个类别中的 978 个属性提示（且可扩展）转化为 FR 空间中的锚点，且无需额外成本。我们并不假设这种迁移必然有效：通过一个人脸验证协议对其进行测量，且仅更改适配器的消融实验可分离出其贡献。并非每个概念都能存活，因为 FR 模型通过丢弃那些在跨因素验证身份时必须忽略的因素来获得其不变性。一种无标签可检测性度量将每个概念在 FR 空间中的可分性与在 VLM 空间中的可分性进行比较，其中可检测性最高的 100 个概念构成模型可读的语义签名，该签名比完整词汇表能更好地区分身份。我们覆盖了四个 FR 主干网络和两个 VLM 编码器；EXPL-FR 无需访问模型架构，支持身份级、逐图像和差分解释。我们在三种监督设置下对属性级审计进行基准测试：人工标签（当前实践）、VLM 伪标签，以及我们完全由提示驱动的审计，并与真实验证行为进行比较。在无标签条件下，提示驱动的审计根据测得的按族裔划分的 RFW 误差对四个 FR 模型进行排序，并根据真实的验证代价对受控属性变化进行排序。

English

Deep face recognition (FR) models reach near-saturated accuracy but remain opaque: a practitioner cannot ask which semantic attributes a similarity score relied upon. EXPL-FR answers this inside the FR model's own embedding space. A lightweight adapter aligns a vision-language model's (VLM) image encoder with the frozen FR space, trained on face images alone and never on text. Because the VLM's encoders share one space, the same adapter applies to the text encoder, turning 978 attribute prompts in 22 categories, also extendable, into FR-space anchors at no extra cost. We do not assume this transfer works: a face-verification protocol measures it, and an ablation changing only the adapter isolates its contribution. Not every concept survives, because an FR model earns its invariances by discarding the factors it must verify identities across. A label-free detectability measure compares each concept's separability in FR space against the VLM space, and the 100 most detectable form the model's readable semantic signature, which separates identities better than the full vocabulary. We cover four FR backbones and two VLM encoders, EXPL-FR needs no architecture access, and supports identity-level, per-image, and differential explanations. We benchmark attribute-level auditing under three supervision settings, human labels (current practice), VLM pseudo-labels, and our fully prompt-driven audit, against real verification behavior. With no labels, the prompt-driven audit ranks four FR models by their measured per-ethnicity RFW errors and ranks controlled attribute changes by their true verification cost.