Click-Gaussian：互動式分割至任意3D高斯函數

Click-Gaussian: Interactive Segmentation to Any 3D Gaussians

July 16, 2024

作者: Seokhun Choi, Hyeonseop Song, Jaechul Kim, Taehyeong Kim, Hoseok Do

cs.AI

摘要

互動式三維高斯分割為實時操作三維場景提供了絕佳機會，這要歸功於三維高斯飛灰的實時渲染能力。然而，目前的方法存在處理嘈雜分割輸出的耗時後處理問題。此外，它們難以提供詳細的分割，而這對於細粒度操作三維場景至關重要。在本研究中，我們提出了Click-Gaussian，它學習了兩級粒度的可區分特徵場，有助於在無需耗時後處理的情況下進行分割。我們深入探討了由於從三維場景獨立獲得的二維分割而導致的特徵場不一致所帶來的挑戰。當跨視圖的二維分割結果，即三維分割的主要線索，存在衝突時，三維分割的準確性會下降。為了克服這些問題，我們提出了全局特徵引導學習（GFL）。GFL從跨視圖的嘈雜二維分割中構建全局特徵候選群，這有助於在訓練三維高斯特徵時平滑噪聲。我們的方法每次點擊運行時間為10毫秒，比先前方法快15到130倍，同時顯著提高了分割準確性。我們的項目頁面位於https://seokhunchoi.github.io/Click-Gaussian。

English

Interactive segmentation of 3D Gaussians opens a great opportunity for real-time manipulation of 3D scenes thanks to the real-time rendering capability of 3D Gaussian Splatting. However, the current methods suffer from time-consuming post-processing to deal with noisy segmentation output. Also, they struggle to provide detailed segmentation, which is important for fine-grained manipulation of 3D scenes. In this study, we propose Click-Gaussian, which learns distinguishable feature fields of two-level granularity, facilitating segmentation without time-consuming post-processing. We delve into challenges stemming from inconsistently learned feature fields resulting from 2D segmentation obtained independently from a 3D scene. 3D segmentation accuracy deteriorates when 2D segmentation results across the views, primary cues for 3D segmentation, are in conflict. To overcome these issues, we propose Global Feature-guided Learning (GFL). GFL constructs the clusters of global feature candidates from noisy 2D segments across the views, which smooths out noises when training the features of 3D Gaussians. Our method runs in 10 ms per click, 15 to 130 times as fast as the previous methods, while also significantly improving segmentation accuracy. Our project page is available at https://seokhunchoi.github.io/Click-Gaussian