分割任意的三維高斯函數

摘要

在輻射場中進行互動式3D分割是一項具吸引力的任務，因為它在3D場景理解和操作中的重要性。然而，現有方法在實現精細、多粒度分割或應對大量計算開銷方面面臨挑戰，從而抑制了實時互動。本文介紹了Segment Any 3D GAussians（SAGA），這是一種新穎的3D互動式分割方法，它將一個2D分割基礎模型與3D高斯飛灰（3DGS）巧妙地結合在一起，後者是輻射場的一項最新突破。SAGA通過精心設計的對比訓練，將分割基礎模型生成的多粒度2D分割結果有效地嵌入到3D高斯點特徵中。對現有基準進行的評估顯示，SAGA能夠與最先進的方法實現競爭性表現。此外，SAGA實現了多粒度分割，並支持各種提示，包括點、塗鴉和2D遮罩。值得注意的是，SAGA可以在毫秒內完成3D分割，與之前的最先進方法相比實現了近1000倍的加速。項目頁面位於https://jumpat.github.io/SAGA。

English

Interactive 3D segmentation in radiance fields is an appealing task since its importance in 3D scene understanding and manipulation. However, existing methods face challenges in either achieving fine-grained, multi-granularity segmentation or contending with substantial computational overhead, inhibiting real-time interaction. In this paper, we introduce Segment Any 3D GAussians (SAGA), a novel 3D interactive segmentation approach that seamlessly blends a 2D segmentation foundation model with 3D Gaussian Splatting (3DGS), a recent breakthrough of radiance fields. SAGA efficiently embeds multi-granularity 2D segmentation results generated by the segmentation foundation model into 3D Gaussian point features through well-designed contrastive training. Evaluation on existing benchmarks demonstrates that SAGA can achieve competitive performance with state-of-the-art methods. Moreover, SAGA achieves multi-granularity segmentation and accommodates various prompts, including points, scribbles, and 2D masks. Notably, SAGA can finish the 3D segmentation within milliseconds, achieving nearly 1000x acceleration compared to previous SOTA. The project page is at https://jumpat.github.io/SAGA.

分割任意的三維高斯函數

Segment Any 3D Gaussians

摘要

Support