分割任意的3D高斯函数。

摘要

在辐射场中进行交互式三维分割是一项吸引人的任务，因为它在三维场景理解和操作中的重要性。然而，现有方法在实现细粒度、多粒度分割或应对大量计算开销方面面临挑战，从而阻碍了实时交互。本文介绍了Segment Any 3D GAussians（SAGA），这是一种新颖的三维交互式分割方法，它将2D分割基础模型与辐射场的最新突破——三维高斯点喷洒（3DGS）巧妙地融合在一起。SAGA通过精心设计的对比训练，将分割基础模型生成的多粒度2D分割结果高效地嵌入到3D高斯点特征中。对现有基准进行评估表明，SAGA能够与最先进的方法实现竞争性表现。此外，SAGA实现了多粒度分割，并支持各种提示，包括点、涂鸦和2D蒙版。值得注意的是，SAGA可以在毫秒内完成三维分割，与之前的最先进方法相比，实现了近1000倍的加速。项目页面位于https://jumpat.github.io/SAGA。

English

Interactive 3D segmentation in radiance fields is an appealing task since its importance in 3D scene understanding and manipulation. However, existing methods face challenges in either achieving fine-grained, multi-granularity segmentation or contending with substantial computational overhead, inhibiting real-time interaction. In this paper, we introduce Segment Any 3D GAussians (SAGA), a novel 3D interactive segmentation approach that seamlessly blends a 2D segmentation foundation model with 3D Gaussian Splatting (3DGS), a recent breakthrough of radiance fields. SAGA efficiently embeds multi-granularity 2D segmentation results generated by the segmentation foundation model into 3D Gaussian point features through well-designed contrastive training. Evaluation on existing benchmarks demonstrates that SAGA can achieve competitive performance with state-of-the-art methods. Moreover, SAGA achieves multi-granularity segmentation and accommodates various prompts, including points, scribbles, and 2D masks. Notably, SAGA can finish the 3D segmentation within milliseconds, achieving nearly 1000x acceleration compared to previous SOTA. The project page is at https://jumpat.github.io/SAGA.

分割任意的3D高斯函数。

Segment Any 3D Gaussians

摘要

Support