FlashSplat：二维到三维的高斯点云分割的最优解

FlashSplat: 2D to 3D Gaussian Splatting Segmentation Solved Optimally

September 12, 2024

作者: Qiuhong Shen, Xingyi Yang, Xinchao Wang

cs.AI

摘要

本研究解决了准确分割3D高斯点云从2D掩模的挑战。传统方法通常依赖于迭代梯度下降来为每个高斯分配唯一标签，导致优化时间长且解决方案次优。相反，我们提出了一个简单而全局最优的3D高斯点云分割求解器。我们方法的核心洞察力在于，通过重建的3D高斯点云场景，2D掩模的渲染基本上是一个线性函数，关于每个高斯的标签。因此，最优标签分配可以通过封闭形式的线性规划来解决。该解决方案利用了高斯点云渲染过程的alpha混合特性，实现了单步优化。通过在目标函数中结合背景偏差，我们的方法在3D分割中展现出对噪声的卓越鲁棒性。值得注意的是，我们的优化在30秒内完成，比最佳现有方法快约50倍。大量实验证明了我们方法在分割各种场景中的效率和鲁棒性，以及在对象去除和修补等下游任务中的卓越性能。演示和代码将在https://github.com/florinshen/FlashSplat 上提供。

English

This study addresses the challenge of accurately segmenting 3D Gaussian Splatting from 2D masks. Conventional methods often rely on iterative gradient descent to assign each Gaussian a unique label, leading to lengthy optimization and sub-optimal solutions. Instead, we propose a straightforward yet globally optimal solver for 3D-GS segmentation. The core insight of our method is that, with a reconstructed 3D-GS scene, the rendering of the 2D masks is essentially a linear function with respect to the labels of each Gaussian. As such, the optimal label assignment can be solved via linear programming in closed form. This solution capitalizes on the alpha blending characteristic of the splatting process for single step optimization. By incorporating the background bias in our objective function, our method shows superior robustness in 3D segmentation against noises. Remarkably, our optimization completes within 30 seconds, about 50times faster than the best existing methods. Extensive experiments demonstrate the efficiency and robustness of our method in segmenting various scenes, and its superior performance in downstream tasks such as object removal and inpainting. Demos and code will be available at https://github.com/florinshen/FlashSplat.