野生高斯：野外的3D高斯点渲染

WildGaussians: 3D Gaussian Splatting in the Wild

July 11, 2024

作者: Jonas Kulhanek, Songyou Peng, Zuzana Kukelova, Marc Pollefeys, Torsten Sattler

cs.AI

摘要

虽然由于其照片级质量，NeRFs主导了3D场景重建领域，但最近出现了3D高斯飘零（3DGS），以实时渲染速度提供类似质量。然而，这两种方法主要擅长处理受控制良好的3D场景，而在野外数据中——其特点是遮挡、动态物体和光照变化——仍然具有挑战性。NeRFs可以通过每个图像的嵌入向量轻松适应这些条件，但3DGS由于其显式表示和缺乏共享参数而面临困难。为了解决这个问题，我们引入了WildGaussians，这是一种处理遮挡和外观变化的新方法，结合了强大的DINO特征，并在3DGS内部集成了外观建模模块，我们的方法实现了最先进的结果。我们展示了WildGaussians与3DGS的实时渲染速度相匹配，同时在处理野外数据方面超越了3DGS和NeRF的基准线，所有这些都在一个简单的架构框架内实现。

English

While the field of 3D scene reconstruction is dominated by NeRFs due to their photorealistic quality, 3D Gaussian Splatting (3DGS) has recently emerged, offering similar quality with real-time rendering speeds. However, both methods primarily excel with well-controlled 3D scenes, while in-the-wild data - characterized by occlusions, dynamic objects, and varying illumination - remains challenging. NeRFs can adapt to such conditions easily through per-image embedding vectors, but 3DGS struggles due to its explicit representation and lack of shared parameters. To address this, we introduce WildGaussians, a novel approach to handle occlusions and appearance changes with 3DGS. By leveraging robust DINO features and integrating an appearance modeling module within 3DGS, our method achieves state-of-the-art results. We demonstrate that WildGaussians matches the real-time rendering speed of 3DGS while surpassing both 3DGS and NeRF baselines in handling in-the-wild data, all within a simple architectural framework.