邁向可解釋的眼底影像基礎模型

摘要

基礎模型用於從大量未標記資料中學習可遷移的表徵，通常透過自監督學習（SSL）達成。然而，許多此類模型依賴於可解釋性有限的架構，這在醫學影像等高風險領域中是一項關鍵問題。我們提出 DualIFM，這是一個在設計上即具可解釋性的基礎模型，其透過 BagNet 骨幹網路中較小的感受野產生忠實反映模型決策過程的類別證據圖。此外，DualIFM 在預訓練期間納入一個二維投影層，使表徵空間得以直接視覺化，提供資料集層級的學習結構視圖，包括具臨床意義的聚類以及潛在的虛假相關性。我們在來自不同來源的逾 80 萬張彩色眼底照片上訓練 DualIFM，以學習適用於不同下游任務的可泛化表徵。我們的模型在分布外資料上提供可解釋的預測，同時其表現可與參數量多 16 倍的 RETFound 相媲美。這些結果表明，大規模自監督預訓練結合內在可解釋性，可為視網膜影像帶來穩健的表徵。程式碼與預訓練模型已公開於 github.com/berenslab/interpretable_FM。

English

Foundation models are used to extract transferable representations from large amounts of unlabeled data, typically via self-supervised learning (SSL). However, many of these models rely on architectures that offer limited interpretability, a critical issue in high-stakes domains such as medical imaging. We propose DualIFM, a foundation model that is interpretable-by-design via a BagNet backbone whose small receptive fields generate class evidence maps that are faithful to the model's decision-making process. Additionally, DualIFM incorporates a 2D projection layer during pretraining that enables direct visualization of the representation space, providing a dataset-level view of the learned structure including meaningful clinical clusters as well as potential spurious correlations. We trained DualIFM on over 800,000 color fundus photographs from various sources to learn generalizable representations for different downstream tasks. Our model achieves performance comparable to RETFound, which has 16times more parameters, while providing interpretable predictions on out-of-distribution data. These results suggest that large-scale SSL pretraining paired with inherent interpretability can lead to robust representations for retinal imaging. Code and pretrained models are available at github.com/berenslab/interpretable_FM.