FlashSplat：二維到三維高斯塗抹分割的最佳解決方案

FlashSplat: 2D to 3D Gaussian Splatting Segmentation Solved Optimally

September 12, 2024

作者: Qiuhong Shen, Xingyi Yang, Xinchao Wang

cs.AI

摘要

本研究解決了從2D遮罩中精確分割3D高斯濺射的挑戰。傳統方法通常依賴迭代梯度下降來為每個高斯分配唯一標籤，導致優化過程冗長且次優。相反，我們提出了一種簡單而全局最優的3D-GS分割求解器。我們方法的核心見解是，通過重建的3D-GS場景，2D遮罩的渲染基本上是一個線性函數，關於每個高斯的標籤。因此，最優標籤分配可以通過閉合形式的線性規劃來解決。這個解決方案利用了濺射過程的alpha混合特性，實現了單步優化。通過將背景偏差納入我們的目標函數，我們的方法在3D分割中展現出優越的魯棒性，對抗噪聲。值得注意的是，我們的優化在30秒內完成，比現有最佳方法快約50倍。大量實驗證明了我們方法在分割各種場景中的效率和魯棒性，以及在對象去除和修補等下游任務中的卓越性能。演示和代碼將在https://github.com/florinshen/FlashSplat 提供。

English

This study addresses the challenge of accurately segmenting 3D Gaussian Splatting from 2D masks. Conventional methods often rely on iterative gradient descent to assign each Gaussian a unique label, leading to lengthy optimization and sub-optimal solutions. Instead, we propose a straightforward yet globally optimal solver for 3D-GS segmentation. The core insight of our method is that, with a reconstructed 3D-GS scene, the rendering of the 2D masks is essentially a linear function with respect to the labels of each Gaussian. As such, the optimal label assignment can be solved via linear programming in closed form. This solution capitalizes on the alpha blending characteristic of the splatting process for single step optimization. By incorporating the background bias in our objective function, our method shows superior robustness in 3D segmentation against noises. Remarkably, our optimization completes within 30 seconds, about 50times faster than the best existing methods. Extensive experiments demonstrate the efficiency and robustness of our method in segmenting various scenes, and its superior performance in downstream tasks such as object removal and inpainting. Demos and code will be available at https://github.com/florinshen/FlashSplat.