SAGS：結構感知 3D 高斯飄點

SAGS: Structure-Aware 3D Gaussian Splatting

April 29, 2024

作者: Evangelos Ververas, Rolandos Alexandros Potamias, Jifei Song, Jiankang Deng, Stefanos Zafeiriou

cs.AI

摘要

隨著 NeRF 的出現，3D 高斯點降（3D-GS）為實時神經渲染打開了一條道路，克服了體積法方法的計算負擔。在 3D-GS 的開創性工作之後，有幾種方法試圖實現可壓縮且高保真性能的替代方案。然而，這些方法採用了與幾何無關的優化方案，忽略了場景固有的 3D 結構，從而限制了表達能力和表現質量，導致各種浮點和瑕疵。在這項工作中，我們提出了一種結構感知高斯點降方法（SAGS），它隱式編碼了場景的幾何結構，反映了最先進的渲染性能，並在基準新視角合成數據集上降低了存儲需求。SAGS 基於一種本地-全局圖表示，有助於學習複雜場景，並強制實施保持場景幾何的有意義的點位移。此外，我們引入了 SAGS 的輕量級版本，使用一種簡單而有效的中點插值方案，展示了一種緊湊的場景表示，無需依賴任何壓縮策略即可實現高達 24 倍的尺寸減小。在多個基準數據集上進行的大量實驗表明，與最先進的 3D-GS 方法相比，SAGS 在渲染質量和模型大小方面具有優越性。此外，我們展示了我們的結構感知方法可以有效地緩解以往方法的浮點瑕疵和不規則失真，同時獲得精確的深度圖。項目頁面：https://eververas.github.io/SAGS/。

English

Following the advent of NeRFs, 3D Gaussian Splatting (3D-GS) has paved the way to real-time neural rendering overcoming the computational burden of volumetric methods. Following the pioneering work of 3D-GS, several methods have attempted to achieve compressible and high-fidelity performance alternatives. However, by employing a geometry-agnostic optimization scheme, these methods neglect the inherent 3D structure of the scene, thereby restricting the expressivity and the quality of the representation, resulting in various floating points and artifacts. In this work, we propose a structure-aware Gaussian Splatting method (SAGS) that implicitly encodes the geometry of the scene, which reflects to state-of-the-art rendering performance and reduced storage requirements on benchmark novel-view synthesis datasets. SAGS is founded on a local-global graph representation that facilitates the learning of complex scenes and enforces meaningful point displacements that preserve the scene's geometry. Additionally, we introduce a lightweight version of SAGS, using a simple yet effective mid-point interpolation scheme, which showcases a compact representation of the scene with up to 24times size reduction without the reliance on any compression strategies. Extensive experiments across multiple benchmark datasets demonstrate the superiority of SAGS compared to state-of-the-art 3D-GS methods under both rendering quality and model size. Besides, we demonstrate that our structure-aware method can effectively mitigate floating artifacts and irregular distortions of previous methods while obtaining precise depth maps. Project page https://eververas.github.io/SAGS/.