重新审视显著图：构建认知对齐的解释性分类与评估框架

Rethinking Saliency Maps: A Cognitive Human Aligned Taxonomy and Evaluation Framework for Explanations

November 17, 2025

作者: Yehonatan Elisha, Seffi Cohen, Oren Barkan, Noam Koenigstein

cs.AI

摘要

显著性图谱虽被广泛用于深度学习的可视化解释，但其预期目标与多样化用户需求之间始终缺乏共识。这种模糊性阻碍了解释方法的有效评估与实际应用。我们通过引入"参考框架×粒度"（RFxG）分类法来弥合这一鸿沟，该原则性概念框架沿两个基本维度组织显著性解释：参考框架维度区分逐点式解释（"为何有此预测？"）与对比式解释（"为何是这个而非其他？"）；粒度维度涵盖从细粒度类别级（如"为何是哈士奇？"）到粗粒度组群级（如"为何是犬科？"）的解释层次。借助RFxG视角，我们揭示了现有评估指标的关键局限——这些指标过度侧重逐点忠实性，却忽视了对比推理与语义粒度。为系统评估RFxG双维度的解释质量，我们提出四项新颖的忠实度指标。该综合评估框架将指标应用于十种前沿显著性方法、四种模型架构及三个数据集。通过推动向用户意图驱动的评估范式转变，本研究不仅为开发可视化解释奠定了概念基础，更提供了实用工具，确保解释结果既忠实于模型内在行为，又能与人类认知和探究的复杂性实现有意义对接。

English

Saliency maps are widely used for visual explanations in deep learning, but a fundamental lack of consensus persists regarding their intended purpose and alignment with diverse user queries. This ambiguity hinders the effective evaluation and practical utility of explanation methods. We address this gap by introducing the Reference-Frame times Granularity (RFxG) taxonomy, a principled conceptual framework that organizes saliency explanations along two essential axes:Reference-Frame: Distinguishing between pointwise ("Why this prediction?") and contrastive ("Why this and not an alternative?") explanations. Granularity: Ranging from fine-grained class-level (e.g., "Why Husky?") to coarse-grained group-level (e.g., "Why Dog?") interpretations. Using the RFxG lens, we demonstrate critical limitations in existing evaluation metrics, which overwhelmingly prioritize pointwise faithfulness while neglecting contrastive reasoning and semantic granularity. To systematically assess explanation quality across both RFxG dimensions, we propose four novel faithfulness metrics. Our comprehensive evaluation framework applies these metrics to ten state-of-the-art saliency methods, four model architectures, and three datasets. By advocating a shift toward user-intent-driven evaluation, our work provides both the conceptual foundation and the practical tools necessary to develop visual explanations that are not only faithful to the underlying model behavior but are also meaningfully aligned with the complexity of human understanding and inquiry.