Click-Gaussian：交互式分割到任意3D高斯函数

Click-Gaussian: Interactive Segmentation to Any 3D Gaussians

July 16, 2024

作者: Seokhun Choi, Hyeonseop Song, Jaechul Kim, Taehyeong Kim, Hoseok Do

cs.AI

摘要

交互式三维高斯分割为实时操作三维场景提供了巨大机遇，得益于三维高斯喷洒的实时渲染能力。然而，当前方法存在处理嘈杂分割输出的耗时后处理问题。此外，它们难以提供重要的用于三维场景精细操作的详细分割。在本研究中，我们提出了Click-Gaussian，学习了两级粒度的可区分特征字段，促进了无需耗时后处理的分割。我们深入探讨了由于独立于三维场景获得的二维分割而导致的学习特征字段不一致所带来的挑战。当跨视图的二维分割结果，即三维分割的主要线索，存在冲突时，三维分割准确性会下降。为了克服这些问题，我们提出了全局特征引导学习（GFL）。GFL从跨视图的嘈杂二维分割中构建全局特征候选群，这有助于在训练三维高斯特征时消除噪声。我们的方法每次点击运行在10毫秒内，速度是先前方法的15到130倍，同时显著提高了分割准确性。我们的项目页面位于https://seokhunchoi.github.io/Click-Gaussian。

English

Interactive segmentation of 3D Gaussians opens a great opportunity for real-time manipulation of 3D scenes thanks to the real-time rendering capability of 3D Gaussian Splatting. However, the current methods suffer from time-consuming post-processing to deal with noisy segmentation output. Also, they struggle to provide detailed segmentation, which is important for fine-grained manipulation of 3D scenes. In this study, we propose Click-Gaussian, which learns distinguishable feature fields of two-level granularity, facilitating segmentation without time-consuming post-processing. We delve into challenges stemming from inconsistently learned feature fields resulting from 2D segmentation obtained independently from a 3D scene. 3D segmentation accuracy deteriorates when 2D segmentation results across the views, primary cues for 3D segmentation, are in conflict. To overcome these issues, we propose Global Feature-guided Learning (GFL). GFL constructs the clusters of global feature candidates from noisy 2D segments across the views, which smooths out noises when training the features of 3D Gaussians. Our method runs in 10 ms per click, 15 to 130 times as fast as the previous methods, while also significantly improving segmentation accuracy. Our project page is available at https://seokhunchoi.github.io/Click-Gaussian