手術 SAM 2：通過高效幀修剪在手術視頻中實時分割任何物件

摘要

在電腦輔助手術中，手術影片分割是一項關鍵任務，對提升手術品質和患者結果至關重要。最近，Segment Anything Model 2（SAM2）框架在影像和影片分割方面展現出優越的進展。然而，由於處理高解析度影像和手術影片中複雜且長程時序動態的高計算需求，SAM2效率不佳。為應對這些挑戰，我們引入了Surgical SAM 2（SurgSAM-2），這是一個先進的模型，利用了SAM2與高效幀修剪（EFP）機制，以促進實時手術影片分割。EFP機制通過動態管理記憶庫，僅選擇保留最具信息量的幀，減少記憶體使用和計算成本，同時保持高分割準確性。我們的廣泛實驗表明，與原始SAM2相比，SurgSAM-2在效率和分割準確性方面均有顯著改善。值得注意的是，SurgSAM-2在與SAM2相比實現了3倍的幀率，同時在與低解析度數據微調後也提供了最新技術的性能。這些進展將SurgSAM-2確立為手術影片分析的領先模型，使得在資源受限環境中進行實時手術影片分割成為可能。

English

Surgical video segmentation is a critical task in computer-assisted surgery and is vital for enhancing surgical quality and patient outcomes. Recently, the Segment Anything Model 2 (SAM2) framework has shown superior advancements in image and video segmentation. However, SAM2 struggles with efficiency due to the high computational demands of processing high-resolution images and complex and long-range temporal dynamics in surgical videos. To address these challenges, we introduce Surgical SAM 2 (SurgSAM-2), an advanced model to utilize SAM2 with an Efficient Frame Pruning (EFP) mechanism, to facilitate real-time surgical video segmentation. The EFP mechanism dynamically manages the memory bank by selectively retaining only the most informative frames, reducing memory usage and computational cost while maintaining high segmentation accuracy. Our extensive experiments demonstrate that SurgSAM-2 significantly improves both efficiency and segmentation accuracy compared to the vanilla SAM2. Remarkably, SurgSAM-2 achieves a 3times FPS compared with SAM2, while also delivering state-of-the-art performance after fine-tuning with lower-resolution data. These advancements establish SurgSAM-2 as a leading model for surgical video analysis, making real-time surgical video segmentation in resource-constrained environments a feasible reality.

手術 SAM 2：通過高效幀修剪在手術視頻中實時分割任何物件

Surgical SAM 2: Real-time Segment Anything in Surgical Video by Efficient Frame Pruning

摘要

Support