HDR-GS：通过高斯飞溅实现的高效1000倍速高动态范围新视角合成

HDR-GS: Efficient High Dynamic Range Novel View Synthesis at 1000x Speed via Gaussian Splatting

May 24, 2024

作者: Yuanhao Cai, Zihao Xiao, Yixun Liang, Yulun Zhang, Xiaokang Yang, Yaoyao Liu, Alan Yuille

cs.AI

摘要

高动态范围（HDR）新视角合成（NVS）旨在利用HDR成像技术从新视角创建逼真图像。渲染的HDR图像捕捉比普通低动态范围（LDR）图像包含更多场景细节的更广泛亮度范围。现有的HDR NVS方法主要基于NeRF。它们存在训练时间长和推理速度慢的问题。本文提出了一个新框架，高动态范围高斯飞溅（HDR-GS），可以高效地渲染新的HDR视角并根据用户输入的曝光时间重建LDR图像。具体来说，我们设计了一个双动态范围（DDR）高斯点云模型，使用球谐函数拟合HDR颜色，并采用基于MLP的色调映射器来渲染LDR颜色。然后，HDR和LDR颜色被输入两个并行可微光栅化（PDR）过程以重建HDR和LDR视角。为了为基于3D高斯飞溅的HDR NVS方法的研究建立数据基础，我们重新校准摄像机参数并计算高斯点云的初始位置。实验证明，我们的HDR-GS在LDR和HDR NVS上分别比最先进的基于NeRF的方法提高了3.84和1.91 dB，同时享有1000倍的推理速度，并且只需要6.3%的训练时间。

English

High dynamic range (HDR) novel view synthesis (NVS) aims to create photorealistic images from novel viewpoints using HDR imaging techniques. The rendered HDR images capture a wider range of brightness levels containing more details of the scene than normal low dynamic range (LDR) images. Existing HDR NVS methods are mainly based on NeRF. They suffer from long training time and slow inference speed. In this paper, we propose a new framework, High Dynamic Range Gaussian Splatting (HDR-GS), which can efficiently render novel HDR views and reconstruct LDR images with a user input exposure time. Specifically, we design a Dual Dynamic Range (DDR) Gaussian point cloud model that uses spherical harmonics to fit HDR color and employs an MLP-based tone-mapper to render LDR color. The HDR and LDR colors are then fed into two Parallel Differentiable Rasterization (PDR) processes to reconstruct HDR and LDR views. To establish the data foundation for the research of 3D Gaussian splatting-based methods in HDR NVS, we recalibrate the camera parameters and compute the initial positions for Gaussian point clouds. Experiments demonstrate that our HDR-GS surpasses the state-of-the-art NeRF-based method by 3.84 and 1.91 dB on LDR and HDR NVS while enjoying 1000x inference speed and only requiring 6.3% training time.