RayGauss：體積高斯射線投影用於逼真的新視角合成

RayGauss: Volumetric Gaussian-Based Ray Casting for Photorealistic Novel View Synthesis

August 6, 2024

作者: Hugo Blanc, Jean-Emmanuel Deschaud, Alexis Paljic

cs.AI

摘要

基於可微分體積渲染的方法在新視角合成方面取得了顯著進展。一方面，創新方法已將神經輻射場（NeRF）網絡替換為具有局部參數化結構的方法，使得在合理時間內能夠生成高質量的渲染。另一方面，一些方法使用了可微分的塗抹（splatting）來取代 NeRF 的光線投射，以使用高斯核快速優化輻射場，實現對場景的精細適應。然而，雖然塗抹實現了快速渲染，但容易產生明顯可見的瑕疵。 我們的工作填補了這一空白，提供了對發射輻射 c 和密度 {\sigma} 進行物理上一致的公式化，使用與球形高斯/調和相關聯的高斯函數進行全頻色度表示。我們還引入了一種方法，通過一種集成輻射場的算法，利用 BVH 結構逐層進行不規則分佈高斯的可微分光線投射。這使得我們的方法能夠對場景進行精細適應，同時避免塗抹的瑕疵。因此，我們實現了比最先進技術更優秀的渲染質量，同時保持合理的訓練時間，並在 Blender 數據集上實現了每秒 25 幀的推理速度。項目頁面連結，包含視頻和代碼：https://raygauss.github.io/

English

Differentiable volumetric rendering-based methods made significant progress in novel view synthesis. On one hand, innovative methods have replaced the Neural Radiance Fields (NeRF) network with locally parameterized structures, enabling high-quality renderings in a reasonable time. On the other hand, approaches have used differentiable splatting instead of NeRF's ray casting to optimize radiance fields rapidly using Gaussian kernels, allowing for fine adaptation to the scene. However, differentiable ray casting of irregularly spaced kernels has been scarcely explored, while splatting, despite enabling fast rendering times, is susceptible to clearly visible artifacts. Our work closes this gap by providing a physically consistent formulation of the emitted radiance c and density {\sigma}, decomposed with Gaussian functions associated with Spherical Gaussians/Harmonics for all-frequency colorimetric representation. We also introduce a method enabling differentiable ray casting of irregularly distributed Gaussians using an algorithm that integrates radiance fields slab by slab and leverages a BVH structure. This allows our approach to finely adapt to the scene while avoiding splatting artifacts. As a result, we achieve superior rendering quality compared to the state-of-the-art while maintaining reasonable training times and achieving inference speeds of 25 FPS on the Blender dataset. Project page with videos and code: https://raygauss.github.io/