Splatfacto-W：一個 Nerfstudio 實作的高斯飛濺技術，適用於無限制的照片集合

Splatfacto-W: A Nerfstudio Implementation of Gaussian Splatting for Unconstrained Photo Collections

July 17, 2024

作者: Congrong Xu, Justin Kerr, Angjoo Kanazawa

cs.AI

摘要

從性野外圖像集合中的新視角合成仍然是一項重要且具挑戰性的任務，這是由於光度變化和瞬時遮擋物使準確場景重建變得複雜。先前的方法通過在神經輻射場（NeRFs）中集成每幅圖像外觀特徵嵌入來應對這些問題。儘管3D高斯飛濺（3DGS）提供更快的訓練和實時渲染，但要將其適應於非受限制的圖像集合是非常困難的，這是由於架構差異顯著。在本文中，我們介紹了Splatfacto-W，一種方法，它將每個高斯神經顏色特徵和每幅圖像外觀嵌入集成到光柵化過程中，並使用基於球面調和的背景模型來表示不同的光度外觀並更好地描述背景。我們的關鍵貢獻包括潛在外觀建模、高效的瞬時物體處理以及精確的背景建模。Splatfacto-W在野外情境中提供了高質量、實時的新視角合成，改善了場景一致性。我們的方法將峰值信噪比（PSNR）平均提高了5.3 dB，比3DGS提高了150倍的訓練速度，並實現了與3DGS相似的渲染速度。額外的視頻結果和代碼已集成到Nerfstudio中，可在https://kevinxu02.github.io/splatfactow/獲得。

English

Novel view synthesis from unconstrained in-the-wild image collections remains a significant yet challenging task due to photometric variations and transient occluders that complicate accurate scene reconstruction. Previous methods have approached these issues by integrating per-image appearance features embeddings in Neural Radiance Fields (NeRFs). Although 3D Gaussian Splatting (3DGS) offers faster training and real-time rendering, adapting it for unconstrained image collections is non-trivial due to the substantially different architecture. In this paper, we introduce Splatfacto-W, an approach that integrates per-Gaussian neural color features and per-image appearance embeddings into the rasterization process, along with a spherical harmonics-based background model to represent varying photometric appearances and better depict backgrounds. Our key contributions include latent appearance modeling, efficient transient object handling, and precise background modeling. Splatfacto-W delivers high-quality, real-time novel view synthesis with improved scene consistency in in-the-wild scenarios. Our method improves the Peak Signal-to-Noise Ratio (PSNR) by an average of 5.3 dB compared to 3DGS, enhances training speed by 150 times compared to NeRF-based methods, and achieves a similar rendering speed to 3DGS. Additional video results and code integrated into Nerfstudio are available at https://kevinxu02.github.io/splatfactow/.