高斯點降需要SFM初始化嗎？

Does Gaussian Splatting need SFM Initialization?

April 18, 2024

作者: Yalda Foroutan, Daniel Rebain, Kwang Moo Yi, Andrea Tagliasacchi

cs.AI

摘要

最近，3D 高斯點擴散被廣泛應用於場景重建和新視角合成，因其高質量的結果和與硬體光柵化的兼容性。儘管高斯點擴散具有優勢，但其對由運動結構（SFM）算法進行高質量點雲初始化的依賴是一個需要克服的重要限制。為此，我們研究了用於高斯點擴散的各種初始化策略，並探討如何利用神經輻射場（NeRF）的體積重建來繞過對SFM數據的依賴。我們的研究結果表明，如果精心設計，隨機初始化可以表現得更好，通過應用改進的初始化策略和從低成本NeRF模型中提取結構，可以實現與SFM初始化獲得的等效結果，甚至有時更優。

English

3D Gaussian Splatting has recently been embraced as a versatile and effective method for scene reconstruction and novel view synthesis, owing to its high-quality results and compatibility with hardware rasterization. Despite its advantages, Gaussian Splatting's reliance on high-quality point cloud initialization by Structure-from-Motion (SFM) algorithms is a significant limitation to be overcome. To this end, we investigate various initialization strategies for Gaussian Splatting and delve into how volumetric reconstructions from Neural Radiance Fields (NeRF) can be utilized to bypass the dependency on SFM data. Our findings demonstrate that random initialization can perform much better if carefully designed and that by employing a combination of improved initialization strategies and structure distillation from low-cost NeRF models, it is possible to achieve equivalent results, or at times even superior, to those obtained from SFM initialization.