智能结肠镜技术前沿

Frontiers in Intelligent Colonoscopy

October 22, 2024

作者: Ge-Peng Ji, Jingyi Liu, Peng Xu, Nick Barnes, Fahad Shahbaz Khan, Salman Khan, Deng-Ping Fan

cs.AI

摘要

结肠镜目前是结直肠癌最敏感的筛查方法之一。本研究探讨智能结肠镜技术的前沿及其对多模态医疗应用的潜在影响。为实现这一目标，我们首先通过四项结肠镜场景感知任务评估当前以数据为中心和以模型为中心的景观，包括分类、检测、分割和视觉-语言理解。这一评估使我们能够识别领域特定挑战，并揭示结肠镜中的多模态研究仍然值得进一步探索。为迎接即将到来的多模态时代，我们建立了三个基础性倡议：一个大规模多模态指令调整数据集 ColonINST，一个专为结肠镜设计的多模态语言模型 ColonGPT，以及一个多模态基准。为促进对这一快速发展领域的持续监测，我们提供了一个用于获取最新更新的公共网站：https://github.com/ai4colonoscopy/IntelliScope。

English

Colonoscopy is currently one of the most sensitive screening methods for colorectal cancer. This study investigates the frontiers of intelligent colonoscopy techniques and their prospective implications for multimodal medical applications. With this goal, we begin by assessing the current data-centric and model-centric landscapes through four tasks for colonoscopic scene perception, including classification, detection, segmentation, and vision-language understanding. This assessment enables us to identify domain-specific challenges and reveals that multimodal research in colonoscopy remains open for further exploration. To embrace the coming multimodal era, we establish three foundational initiatives: a large-scale multimodal instruction tuning dataset ColonINST, a colonoscopy-designed multimodal language model ColonGPT, and a multimodal benchmark. To facilitate ongoing monitoring of this rapidly evolving field, we provide a public website for the latest updates: https://github.com/ai4colonoscopy/IntelliScope.