高斯霜：可编辑的复杂辐射场与实时渲染

Gaussian Frosting: Editable Complex Radiance Fields with Real-Time Rendering

March 21, 2024

作者: Antoine Guédon, Vincent Lepetit

cs.AI

摘要

我们提出了高斯糖霜（Gaussian Frosting）, 一种新颖的基于网格的表示方法，用于实时渲染和编辑复杂三维效果。我们的方法建立在最近的三维高斯飞溅（3D Gaussian Splatting）框架之上，该框架优化一组三维高斯函数以近似从图像中提取的辐射场。我们首先在优化过程中从高斯函数中提取基础网格，然后构建和细化一个自适应的高斯函数层，围绕网格以可变厚度捕捉表面附近的细节和体积效果，如头发或草。我们将这一层称为高斯糖霜，因为它类似于蛋糕上的一层糖霜。材料越模糊，糖霜越厚。我们还引入了高斯函数的参数化，以强制它们保持在糖霜层内，并在变形、重新缩放、编辑或动画化网格时自动调整其参数。我们的表示方法允许使用高斯飞溅进行高效渲染，同时通过修改基础网格进行编辑和动画制作。我们在各种合成和真实场景上展示了我们方法的有效性，并表明其优于现有的基于表面的方法。我们将发布我们的代码和一个基于网络的查看器作为额外贡献。我们的项目页面如下: https://anttwo.github.io/frosting/

English

We propose Gaussian Frosting, a novel mesh-based representation for high-quality rendering and editing of complex 3D effects in real-time. Our approach builds on the recent 3D Gaussian Splatting framework, which optimizes a set of 3D Gaussians to approximate a radiance field from images. We propose first extracting a base mesh from Gaussians during optimization, then building and refining an adaptive layer of Gaussians with a variable thickness around the mesh to better capture the fine details and volumetric effects near the surface, such as hair or grass. We call this layer Gaussian Frosting, as it resembles a coating of frosting on a cake. The fuzzier the material, the thicker the frosting. We also introduce a parameterization of the Gaussians to enforce them to stay inside the frosting layer and automatically adjust their parameters when deforming, rescaling, editing or animating the mesh. Our representation allows for efficient rendering using Gaussian splatting, as well as editing and animation by modifying the base mesh. We demonstrate the effectiveness of our method on various synthetic and real scenes, and show that it outperforms existing surface-based approaches. We will release our code and a web-based viewer as additional contributions. Our project page is the following: https://anttwo.github.io/frosting/