SAGS：结构感知 3D 高斯飘带

SAGS: Structure-Aware 3D Gaussian Splatting

April 29, 2024

作者: Evangelos Ververas, Rolandos Alexandros Potamias, Jifei Song, Jiankang Deng, Stefanos Zafeiriou

cs.AI

摘要

随着NeRF的出现，3D高斯喷洒（3D-GS）为实时神经渲染铺平了道路，克服了体积方法的计算负担。在3D-GS的开创性工作之后，有几种方法尝试实现可压缩且高保真性能的替代方案。然而，通过采用几何无关的优化方案，这些方法忽视了场景固有的3D结构，从而限制了表达能力和表示的质量，导致各种浮点和伪影。在这项工作中，我们提出了一种结构感知的高斯喷洒方法（SAGS），它隐式地编码了场景的几何结构，反映了最先进的渲染性能，并在基准新视角合成数据集上降低了存储需求。SAGS基于本地-全局图表示，有助于学习复杂场景，并强制执行保留场景几何的有意义的点位移。此外，我们引入了SAGS的轻量级版本，使用简单而有效的中点插值方案，展示了一种紧凑的场景表示，可实现高达24倍的尺寸缩减，而无需依赖任何压缩策略。在多个基准数据集上进行的大量实验表明，与最先进的3D-GS方法相比，SAGS在渲染质量和模型大小方面具有优越性。此外，我们证明了我们的结构感知方法可以有效地减轻以往方法的浮点伪影和不规则失真，同时获得精确的深度图。项目页面https://eververas.github.io/SAGS/。

English

Following the advent of NeRFs, 3D Gaussian Splatting (3D-GS) has paved the way to real-time neural rendering overcoming the computational burden of volumetric methods. Following the pioneering work of 3D-GS, several methods have attempted to achieve compressible and high-fidelity performance alternatives. However, by employing a geometry-agnostic optimization scheme, these methods neglect the inherent 3D structure of the scene, thereby restricting the expressivity and the quality of the representation, resulting in various floating points and artifacts. In this work, we propose a structure-aware Gaussian Splatting method (SAGS) that implicitly encodes the geometry of the scene, which reflects to state-of-the-art rendering performance and reduced storage requirements on benchmark novel-view synthesis datasets. SAGS is founded on a local-global graph representation that facilitates the learning of complex scenes and enforces meaningful point displacements that preserve the scene's geometry. Additionally, we introduce a lightweight version of SAGS, using a simple yet effective mid-point interpolation scheme, which showcases a compact representation of the scene with up to 24times size reduction without the reliance on any compression strategies. Extensive experiments across multiple benchmark datasets demonstrate the superiority of SAGS compared to state-of-the-art 3D-GS methods under both rendering quality and model size. Besides, we demonstrate that our structure-aware method can effectively mitigate floating artifacts and irregular distortions of previous methods while obtaining precise depth maps. Project page https://eververas.github.io/SAGS/.