RayGauss：基于体积高斯的光线投射用于逼真的新颖视图合成
RayGauss: Volumetric Gaussian-Based Ray Casting for Photorealistic Novel View Synthesis
August 6, 2024
作者: Hugo Blanc, Jean-Emmanuel Deschaud, Alexis Paljic
cs.AI
摘要
基于可微体积渲染的方法在新视角合成方面取得了显著进展。一方面，创新方法已经用局部参数化结构取代了神经辐射场（NeRF）网络，使得在合理时间内能够实现高质量渲染。另一方面，一些方法采用了可微喷洒技术，而不是NeRF的光线投射，通过高斯核快速优化辐射场，实现对场景的精细适应。然而，尽管喷洒技术能够实现快速渲染，但对明显可见的伪影很敏感，而对于不规则间隔核的可微光线投射研究却鲜有涉及。
我们的工作通过提供发射辐射c和密度σ的物理一致性公式，将其分解为与球面高斯/谐波相关的高斯函数，实现全频色度表示。我们还引入了一种方法，通过一种集成辐射场的分块逐层进行的算法，并利用BVH结构，实现了对不规则分布高斯的可微光线投射。这使得我们的方法能够在避免喷洒技术伪影的同时，对场景进行精细调整。因此，我们在保持合理训练时间的同时，实现了比现有技术更优越的渲染质量，并在Blender数据集上实现了每秒25帧的推理速度。项目页面包含视频和代码：https://raygauss.github.io/
English
Differentiable volumetric rendering-based methods made significant progress
in novel view synthesis. On one hand, innovative methods have replaced the
Neural Radiance Fields (NeRF) network with locally parameterized structures,
enabling high-quality renderings in a reasonable time. On the other hand,
approaches have used differentiable splatting instead of NeRF's ray casting to
optimize radiance fields rapidly using Gaussian kernels, allowing for fine
adaptation to the scene. However, differentiable ray casting of irregularly
spaced kernels has been scarcely explored, while splatting, despite enabling
fast rendering times, is susceptible to clearly visible artifacts.
Our work closes this gap by providing a physically consistent formulation of
the emitted radiance c and density {\sigma}, decomposed with Gaussian functions
associated with Spherical Gaussians/Harmonics for all-frequency colorimetric
representation. We also introduce a method enabling differentiable ray casting
of irregularly distributed Gaussians using an algorithm that integrates
radiance fields slab by slab and leverages a BVH structure. This allows our
approach to finely adapt to the scene while avoiding splatting artifacts. As a
result, we achieve superior rendering quality compared to the state-of-the-art
while maintaining reasonable training times and achieving inference speeds of
25 FPS on the Blender dataset. Project page with videos and code:
https://raygauss.github.io/