高斯霜：具有即時渲染的可編輯複雜輻射場

Gaussian Frosting: Editable Complex Radiance Fields with Real-Time Rendering

March 21, 2024

作者: Antoine Guédon, Vincent Lepetit

cs.AI

摘要

我們提出了高斯霜(Gaussian Frosting)，這是一種新穎的基於網格的表示法，可用於實時高質量渲染和編輯複雜的3D效果。我們的方法建立在最近的3D高斯濺射框架之上，該框架優化一組3D高斯函數，以從影像中近似辐射場。我們首先提出在優化過程中從高斯函數中提取基礎網格，然後在網格周圍建立和精細調節一層可變厚度的高斯函數，以更好地捕捉靠近表面的細節和體積效應，如頭髮或草。我們稱這一層為高斯霜，因為它類似於蛋糕上的一層霜飾。材料越模糊，霜飾越厚。我們還引入了高斯函數的參數化，以強制它們保持在霜飾層內，並在變形、重新縮放、編輯或動畫化網格時自動調整其參數。我們的表示法允許使用高斯濺射進行高效渲染，同時通過修改基礎網格進行編輯和動畫製作。我們在各種合成和真實場景中展示了我們方法的有效性，並表明它優於現有的基於表面的方法。我們將釋出代碼和基於Web的查看器作為額外貢獻。我們的項目頁面如下：https://anttwo.github.io/frosting/

English

We propose Gaussian Frosting, a novel mesh-based representation for high-quality rendering and editing of complex 3D effects in real-time. Our approach builds on the recent 3D Gaussian Splatting framework, which optimizes a set of 3D Gaussians to approximate a radiance field from images. We propose first extracting a base mesh from Gaussians during optimization, then building and refining an adaptive layer of Gaussians with a variable thickness around the mesh to better capture the fine details and volumetric effects near the surface, such as hair or grass. We call this layer Gaussian Frosting, as it resembles a coating of frosting on a cake. The fuzzier the material, the thicker the frosting. We also introduce a parameterization of the Gaussians to enforce them to stay inside the frosting layer and automatically adjust their parameters when deforming, rescaling, editing or animating the mesh. Our representation allows for efficient rendering using Gaussian splatting, as well as editing and animation by modifying the base mesh. We demonstrate the effectiveness of our method on various synthetic and real scenes, and show that it outperforms existing surface-based approaches. We will release our code and a web-based viewer as additional contributions. Our project page is the following: https://anttwo.github.io/frosting/