穩健的高斯點陣化

Robust Gaussian Splatting

April 5, 2024

作者: François Darmon, Lorenzo Porzi, Samuel Rota-Bulò, Peter Kontschieder

cs.AI

摘要

本文探討了3D高斯點降（3DGS）中常見的錯誤來源，包括模糊、不完美的相機姿勢和顏色不一致性，旨在提高其在實際應用中（如從手機拍攝的重建）的魯棒性。我們的主要貢獻在於將運動模糊建模為相機姿勢上的高斯分佈，從而使我們能夠以統一的方式處理相機姿勢的精化和運動模糊校正。此外，我們提出了用於處理焦外模糊補償以及解決由環境光、陰影或相機相關因素（如不同的白平衡設置）引起的顏色不一致性的機制。我們提出的解決方案與3DGS公式無縫集成，同時保持其在訓練效率和渲染速度方面的優勢。我們在相關基準數據集（包括Scannet++和Deblur-NeRF）上進行了實驗驗證，獲得了最先進的結果，從而相對於相關基準線實現了一致的改進。

English

In this paper, we address common error sources for 3D Gaussian Splatting (3DGS) including blur, imperfect camera poses, and color inconsistencies, with the goal of improving its robustness for practical applications like reconstructions from handheld phone captures. Our main contribution involves modeling motion blur as a Gaussian distribution over camera poses, allowing us to address both camera pose refinement and motion blur correction in a unified way. Additionally, we propose mechanisms for defocus blur compensation and for addressing color in-consistencies caused by ambient light, shadows, or due to camera-related factors like varying white balancing settings. Our proposed solutions integrate in a seamless way with the 3DGS formulation while maintaining its benefits in terms of training efficiency and rendering speed. We experimentally validate our contributions on relevant benchmark datasets including Scannet++ and Deblur-NeRF, obtaining state-of-the-art results and thus consistent improvements over relevant baselines.