稳健的高斯点阵化

Robust Gaussian Splatting

April 5, 2024

作者: François Darmon, Lorenzo Porzi, Samuel Rota-Bulò, Peter Kontschieder

cs.AI

摘要

本文讨论了3D高斯点云投影（3DGS）中的常见误差源，包括模糊、不完美的相机姿势和颜色不一致性，旨在提高其在实际应用中的鲁棒性，如从手持手机拍摄中进行的重建。我们的主要贡献在于将运动模糊建模为相机姿势上的高斯分布，使我们能够以统一的方式解决相机姿势的精化和运动模糊校正。此外，我们提出了用于处理焦外模糊补偿以及解决由环境光、阴影或由于相机相关因素（如不同的白平衡设置）引起的颜色不一致性的机制。我们提出的解决方案与3DGS公式无缝集成，同时保持其在训练效率和渲染速度方面的优势。我们在相关基准数据集（包括Scannet++和Deblur-NeRF）上对我们的贡献进行了实验证实，获得了最先进的结果，从而相对于相关基准线实现了一致的改进。

English

In this paper, we address common error sources for 3D Gaussian Splatting (3DGS) including blur, imperfect camera poses, and color inconsistencies, with the goal of improving its robustness for practical applications like reconstructions from handheld phone captures. Our main contribution involves modeling motion blur as a Gaussian distribution over camera poses, allowing us to address both camera pose refinement and motion blur correction in a unified way. Additionally, we propose mechanisms for defocus blur compensation and for addressing color in-consistencies caused by ambient light, shadows, or due to camera-related factors like varying white balancing settings. Our proposed solutions integrate in a seamless way with the 3DGS formulation while maintaining its benefits in terms of training efficiency and rendering speed. We experimentally validate our contributions on relevant benchmark datasets including Scannet++ and Deblur-NeRF, obtaining state-of-the-art results and thus consistent improvements over relevant baselines.