MIRAGE：面向全面视网膜OCT图像分析的多模态基础模型与基准测试

摘要

人工智能（AI）已成为辅助临床医生分析眼科影像（如光学相干断层扫描，OCT）的重要工具。然而，开发AI模型通常需要大量标注，且现有模型在独立、未见过的数据上表现欠佳。基础模型（FMs）——在大量未标注数据集上训练的大型AI模型——已显示出克服这些挑战的潜力。尽管如此，现有的眼科基础模型缺乏广泛验证，尤其是在分割任务上，且仅关注单一成像模式。在此背景下，我们提出了MIRAGE，一种新型多模态基础模型，用于分析OCT和扫描激光眼底成像（SLO）图像。此外，我们提出了一个新的评估基准，包含OCT/SLO分类和分割任务。与通用和专用基础模型及分割方法的比较表明，MIRAGE在两类任务中均表现出色，凸显了其作为开发稳健视网膜OCT图像分析AI系统基础的适用性。MIRAGE及评估基准均已公开：https://github.com/j-morano/MIRAGE。

English

Artificial intelligence (AI) has become a fundamental tool for assisting clinicians in analyzing ophthalmic images, such as optical coherence tomography (OCT). However, developing AI models often requires extensive annotation, and existing models tend to underperform on independent, unseen data. Foundation models (FMs), large AI models trained on vast unlabeled datasets, have shown promise in overcoming these challenges. Nonetheless, available FMs for ophthalmology lack extensive validation, especially for segmentation tasks, and focus on a single imaging modality. In this context, we propose MIRAGE, a novel multimodal FM for the analysis of OCT and scanning laser ophthalmoscopy (SLO) images. Additionally, we propose a new evaluation benchmark with OCT/SLO classification and segmentation tasks. The comparison with general and specialized FMs and segmentation methods shows the superiority of MIRAGE in both types of tasks, highlighting its suitability as a basis for the development of robust AI systems for retinal OCT image analysis. Both MIRAGE and the evaluation benchmark are publicly available: https://github.com/j-morano/MIRAGE.

MIRAGE：面向全面视网膜OCT图像分析的多模态基础模型与基准测试

MIRAGE: Multimodal foundation model and benchmark for comprehensive retinal OCT image analysis

摘要

Support