SuGaR：表面對齊高斯飛濺用於高效3D網格重建和高質量網格渲染

SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Rendering

November 21, 2023

作者: Antoine Guédon, Vincent Lepetit

cs.AI

摘要

我們提出了一種方法，可以從3D高斯濺射中精確且極快速地提取網格。高斯濺射最近變得非常流行，因為它提供了逼真的渲染效果，而且訓練速度顯著快於神經光子反射(NeRFs)。然而，從數百萬個微小的3D高斯濺射中提取網格是具有挑戰性的，因為這些高斯濺射在優化後往往是無序的，且迄今為止還沒有提出相應的方法。我們的第一個關鍵貢獻是引入一個正則化項，鼓勵高斯濺射與場景表面良好對齊。然後，我們介紹了一種利用這種對齊性的方法，通過泊松重建從高斯濺射中提取網格，這種方法快速、可擴展，並且保留細節，與通常用於從神經SDF中提取網格的Marching Cubes算法形成對比。最後，我們引入了一種可選的細化策略，將高斯濺射綁定到網格表面，並通過高斯濺射渲染同時優化這些高斯濺射和網格。這使得可以通過操縱網格而不是高斯濺射本身，使用傳統軟件輕鬆編輯、雕塑、綁定、動畫製作、合成和重新燈光高斯濺射。使用我們的方法，可以在幾分鐘內獲得這樣一個可編輯的網格，而與神經SDF的最新方法相比，後者需要數小時，同時提供更好的渲染質量。

English

We propose a method to allow precise and extremely fast mesh extraction from 3D Gaussian Splatting. Gaussian Splatting has recently become very popular as it yields realistic rendering while being significantly faster to train than NeRFs. It is however challenging to extract a mesh from the millions of tiny 3D gaussians as these gaussians tend to be unorganized after optimization and no method has been proposed so far. Our first key contribution is a regularization term that encourages the gaussians to align well with the surface of the scene. We then introduce a method that exploits this alignment to extract a mesh from the Gaussians using Poisson reconstruction, which is fast, scalable, and preserves details, in contrast to the Marching Cubes algorithm usually applied to extract meshes from Neural SDFs. Finally, we introduce an optional refinement strategy that binds gaussians to the surface of the mesh, and jointly optimizes these Gaussians and the mesh through Gaussian splatting rendering. This enables easy editing, sculpting, rigging, animating, compositing and relighting of the Gaussians using traditional softwares by manipulating the mesh instead of the gaussians themselves. Retrieving such an editable mesh for realistic rendering is done within minutes with our method, compared to hours with the state-of-the-art methods on neural SDFs, while providing a better rendering quality.