MIRAGE：面向全面視網膜OCT影像分析的多模態基礎模型與基準測試

摘要

人工智慧（AI）已成為協助臨床醫師分析眼科影像（如光學相干斷層掃描，OCT）的基本工具。然而，開發AI模型通常需要大量的標註，且現有模型在獨立、未見過的數據上表現往往不佳。基礎模型（FMs）是基於大量未標註數據訓練的大型AI模型，已顯示出克服這些挑戰的潛力。然而，現有的眼科FMs缺乏廣泛的驗證，特別是在分割任務上，並且僅專注於單一影像模式。在此背景下，我們提出了MIRAGE，一種新穎的多模態FM，用於分析OCT和掃描激光眼底鏡（SLO）影像。此外，我們提出了一個新的評估基準，包含OCT/SLO分類和分割任務。與通用和專用FMs及分割方法的比較顯示，MIRAGE在兩類任務中均表現優異，突顯其作為開發用於視網膜OCT影像分析的穩健AI系統基礎的適宜性。MIRAGE及評估基準均已公開提供：https://github.com/j-morano/MIRAGE。

English

Artificial intelligence (AI) has become a fundamental tool for assisting clinicians in analyzing ophthalmic images, such as optical coherence tomography (OCT). However, developing AI models often requires extensive annotation, and existing models tend to underperform on independent, unseen data. Foundation models (FMs), large AI models trained on vast unlabeled datasets, have shown promise in overcoming these challenges. Nonetheless, available FMs for ophthalmology lack extensive validation, especially for segmentation tasks, and focus on a single imaging modality. In this context, we propose MIRAGE, a novel multimodal FM for the analysis of OCT and scanning laser ophthalmoscopy (SLO) images. Additionally, we propose a new evaluation benchmark with OCT/SLO classification and segmentation tasks. The comparison with general and specialized FMs and segmentation methods shows the superiority of MIRAGE in both types of tasks, highlighting its suitability as a basis for the development of robust AI systems for retinal OCT image analysis. Both MIRAGE and the evaluation benchmark are publicly available: https://github.com/j-morano/MIRAGE.

MIRAGE：面向全面視網膜OCT影像分析的多模態基礎模型與基準測試

MIRAGE: Multimodal foundation model and benchmark for comprehensive retinal OCT image analysis

摘要

Support