SAM2Matting: 通用的图像和视频抠图

摘要

尽管图像抠图技术取得了显著进步，但视频抠图仍面临挑战，这源于高级跟踪（需要逐帧理解）与低级抠图（专注于极其精细的细节）之间的固有差距。现有方法依赖昂贵且范围狭窄的视频抠图数据集，这可能限制跨域泛化能力并削弱跟踪的鲁棒性。我们通过SAM2Matting重新思考了这一范式——这是一个从跟踪到抠图的框架，将VOS跟踪器提升为高保真视频抠图系统。具体而言，该框架通过为基础跟踪器（例如SAM2、SAM3）配备区域提议桥接模块和专用抠图头，将任务解耦，使得不受妥协的跟踪器能够处理时间一致性，而抠图组件则负责解析精细细节。值得注意的是，尽管仅在图像上训练，SAM2Matting在视频抠图任务上树立了新的最优性能，支持多种提示类型，保持强时间一致性，并在以人为中心和野外场景中均展现出强大的泛化能力。

English

Despite impressive advances in image matting, video matting remains challenging due to the inherent gap between high-level tracking, which requires frame-wise understanding, and low-level matting, which focuses on extremely fine-grained details. Existing methods attempt this with expensive and narrowly-scoped video matting datasets, which may limit out-of-domain generalization and compromise tracking robustness. We rethink the paradigm with SAM2Matting, a tracker-to-matting framework that advances VOS trackers to high-fidelity video matting. Specifically, it decouples the task by enhancing a foundational tracker (e.g., SAM2, SAM3) with a region-proposal bridge and dedicated matting heads, enabling the uncompromised tracker to handle temporal consistency while the matting components resolve fine-grained details. Notably, despite being trained only on images, SAM2Matting establishes new state-of-the-art performance on video matting, supports diverse prompt types, maintains strong temporal consistency, and demonstrates robust generalization across both human-centric and in-the-wild scenarios.