SuGaR：面对齐的高斯飘粒用于高效的3D网格重建和高质量网格渲染

SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Rendering

November 21, 2023

作者: Antoine Guédon, Vincent Lepetit

cs.AI

摘要

我们提出了一种方法，允许从三维高斯飞溅中精确且极快地提取网格。高斯飞溅最近变得非常流行，因为它在训练速度上比神经辐射场（NeRFs）快得多，并且能产生逼真的渲染效果。然而，从数百万个微小的三维高斯中提取网格是具有挑战性的，因为这些高斯在优化后往往是无序的，迄今为止还没有提出有效的方法。我们的第一个关键贡献是引入了一个正则化项，鼓励高斯与场景表面良好对齐。然后，我们介绍了一种利用这种对齐性的方法，通过泊松重建从高斯中提取网格，这种方法快速、可扩展，并且保留细节，与通常用于从神经SDF中提取网格的Marching Cubes算法形成对比。最后，我们引入了一个可选的细化策略，将高斯绑定到网格表面，并通过高斯飞溅渲染同时优化这些高斯和网格。这使得通过操纵网格而不是高斯本身，可以使用传统软件轻松编辑、雕刻、绑定、动画制作、合成和重新照明高斯。通过我们的方法，获取用于逼真渲染的可编辑网格仅需几分钟，而使用神经SDF的最先进方法可能需要数小时，同时提供更好的渲染质量。

English

We propose a method to allow precise and extremely fast mesh extraction from 3D Gaussian Splatting. Gaussian Splatting has recently become very popular as it yields realistic rendering while being significantly faster to train than NeRFs. It is however challenging to extract a mesh from the millions of tiny 3D gaussians as these gaussians tend to be unorganized after optimization and no method has been proposed so far. Our first key contribution is a regularization term that encourages the gaussians to align well with the surface of the scene. We then introduce a method that exploits this alignment to extract a mesh from the Gaussians using Poisson reconstruction, which is fast, scalable, and preserves details, in contrast to the Marching Cubes algorithm usually applied to extract meshes from Neural SDFs. Finally, we introduce an optional refinement strategy that binds gaussians to the surface of the mesh, and jointly optimizes these Gaussians and the mesh through Gaussian splatting rendering. This enables easy editing, sculpting, rigging, animating, compositing and relighting of the Gaussians using traditional softwares by manipulating the mesh instead of the gaussians themselves. Retrieving such an editable mesh for realistic rendering is done within minutes with our method, compared to hours with the state-of-the-art methods on neural SDFs, while providing a better rendering quality.