野生高斯：野外的三維高斯點陣化

WildGaussians: 3D Gaussian Splatting in the Wild

July 11, 2024

作者: Jonas Kulhanek, Songyou Peng, Zuzana Kukelova, Marc Pollefeys, Torsten Sattler

cs.AI

摘要

儘管在3D場景重建領域中，由於其逼真的質量，NeRFs佔主導地位，但最近出現了3D高斯飛灑（3DGS），提供了與實時渲染速度相近的相似質量。然而，這兩種方法主要在控制良好的3D場景中表現出色，而在野外數據——以遮蔽、動態物體和不同照明為特徵——仍然具有挑戰性。NeRFs可以通過每幅圖像的嵌入向量輕鬆適應這些條件，但3DGS由於其明確表示和缺乏共享參數而遇到困難。為了解決這個問題，我們引入了WildGaussians，一種處理3DGS中遮蔽和外觀變化的新方法。通過利用強大的DINO特徵並在3DGS內部集成外觀建模模塊，我們的方法實現了最先進的結果。我們展示了WildGaussians與3DGS的實時渲染速度相匹配，同時在處理野外數據方面超越了3DGS和NeRF的基準線，所有這些都在一個簡單的架構框架內實現。

English

While the field of 3D scene reconstruction is dominated by NeRFs due to their photorealistic quality, 3D Gaussian Splatting (3DGS) has recently emerged, offering similar quality with real-time rendering speeds. However, both methods primarily excel with well-controlled 3D scenes, while in-the-wild data - characterized by occlusions, dynamic objects, and varying illumination - remains challenging. NeRFs can adapt to such conditions easily through per-image embedding vectors, but 3DGS struggles due to its explicit representation and lack of shared parameters. To address this, we introduce WildGaussians, a novel approach to handle occlusions and appearance changes with 3DGS. By leveraging robust DINO features and integrating an appearance modeling module within 3DGS, our method achieves state-of-the-art results. We demonstrate that WildGaussians matches the real-time rendering speed of 3DGS while surpassing both 3DGS and NeRF baselines in handling in-the-wild data, all within a simple architectural framework.