用於實時輻射場渲染的3D高斯點塗畫

摘要

輝度場方法最近已經徹底改革了使用多張照片或視頻捕捉的場景的新視角合成。然而，要實現高視覺品質仍需要昂貴的神經網絡進行訓練和渲染，而最近的更快方法不可避免地在速度和品質之間做出取捨。對於無限且完整的場景（而不僅僅是孤立的物體）和 1080p 分辨率渲染，目前沒有任何方法能夠實現實時顯示速率。我們引入了三個關鍵元素，使我們能夠實現最先進的視覺品質，同時保持具有競爭力的訓練時間，並且重要的是實現高質量的實時（>= 30 fps）新視角合成，分辨率為 1080p。首先，從相機校準期間生成的稀疏點開始，我們用保留了連續體積輻射場的理想特性的 3D 高斯函數來表示場景，以進行場景優化，同時避免在空白空間中進行不必要的計算；其次，我們對 3D 高斯函數執行交錯優化/密度控制，特別是優化各向異性協方差以實現對場景的準確表示；第三，我們開發了一種快速的可見性感知渲染算法，支持各向異性飛濺，加速訓練並實現實時渲染。我們在幾個已建立的數據集上展示了最先進的視覺品質和實時渲染。

English

Radiance Field methods have recently revolutionized novel-view synthesis of scenes captured with multiple photos or videos. However, achieving high visual quality still requires neural networks that are costly to train and render, while recent faster methods inevitably trade off speed for quality. For unbounded and complete scenes (rather than isolated objects) and 1080p resolution rendering, no current method can achieve real-time display rates. We introduce three key elements that allow us to achieve state-of-the-art visual quality while maintaining competitive training times and importantly allow high-quality real-time (>= 30 fps) novel-view synthesis at 1080p resolution. First, starting from sparse points produced during camera calibration, we represent the scene with 3D Gaussians that preserve desirable properties of continuous volumetric radiance fields for scene optimization while avoiding unnecessary computation in empty space; Second, we perform interleaved optimization/density control of the 3D Gaussians, notably optimizing anisotropic covariance to achieve an accurate representation of the scene; Third, we develop a fast visibility-aware rendering algorithm that supports anisotropic splatting and both accelerates training and allows realtime rendering. We demonstrate state-of-the-art visual quality and real-time rendering on several established datasets.

用於實時輻射場渲染的3D高斯點塗畫

3D Gaussian Splatting for Real-Time Radiance Field Rendering

摘要

Support