迈向可解释的视网膜眼底图像基础模型

摘要

基础模型用于从大量未标记数据中提取可迁移的表征，通常通过自监督学习（SSL）实现。然而，许多此类模型依赖于可解释性有限的架构，这在医学成像等高风险领域是一个关键问题。我们提出了DualIFM，一种通过BagNet骨干网络实现设计上可解释的基础模型，其小感受野生成忠实于模型决策过程的类别证据图。此外，DualIFM在预训练期间引入了二维投影层，能够直接可视化表征空间，提供数据集级别的学习结构视图，包括有意义的临床聚类以及潜在的虚假相关。我们在来自不同来源的超过80万张彩色眼底照片上训练了DualIFM，以学习适用于不同下游任务的泛化性强的表征。我们的模型取得了与参数数量为其16倍的RETFound相当的性能，同时在分布外数据上提供了可解释的预测。这些结果表明，大规模自监督预训练与固有的可解释性相结合，可以为视网膜影像产生稳健的表征。代码和预训练模型可在github.com/berenslab/interpretable_FM获取。

English

Foundation models are used to extract transferable representations from large amounts of unlabeled data, typically via self-supervised learning (SSL). However, many of these models rely on architectures that offer limited interpretability, a critical issue in high-stakes domains such as medical imaging. We propose DualIFM, a foundation model that is interpretable-by-design via a BagNet backbone whose small receptive fields generate class evidence maps that are faithful to the model's decision-making process. Additionally, DualIFM incorporates a 2D projection layer during pretraining that enables direct visualization of the representation space, providing a dataset-level view of the learned structure including meaningful clinical clusters as well as potential spurious correlations. We trained DualIFM on over 800,000 color fundus photographs from various sources to learn generalizable representations for different downstream tasks. Our model achieves performance comparable to RETFound, which has 16times more parameters, while providing interpretable predictions on out-of-distribution data. These results suggest that large-scale SSL pretraining paired with inherent interpretability can lead to robust representations for retinal imaging. Code and pretrained models are available at github.com/berenslab/interpretable_FM.