GaussianSR：具有2D擴散先驗的3D高斯超分辨率

GaussianSR: 3D Gaussian Super-Resolution with 2D Diffusion Priors

June 14, 2024

作者: Xiqian Yu, Hanxin Zhu, Tianyu He, Zhibo Chen

cs.AI

摘要

從低解析度的輸入視圖實現高解析度新視圖合成（HRNVS）是一項具有挑戰性的任務，原因在於缺乏高解析度的數據。先前的方法優化高解析度的神經輻射場（NeRF）以從低解析度的輸入視圖中獲得，但渲染速度較慢。在這項研究中，我們基於 3D 高斯飛濺（3DGS）的方法，因其能夠以更快的渲染速度生成高質量圖像。為了減輕高解析度合成所需數據的不足，我們提出利用現成的 2D 擴散先驗，通過使用得分蒸餾取得 2D 知識並轉化為 3D。然而，將得分蒸餾直接應用於基於高斯的 3D 超分辨率會導致不必要和冗餘的 3D 高斯基元，這是由生成先驗帶來的隨機性所導致的。為了緩解這個問題，我們引入了兩種簡單而有效的技術來減少得分蒸餾引入的隨機干擾。具體來說，我們 1）通過一種退火策略縮小 SDS 中擴散時間步的範圍；2）在密集化過程中隨機丟棄冗餘的高斯基元。大量實驗表明，我們提出的 GaussainSR 能夠在合成和真實世界數據集上僅使用低解析度輸入即可獲得 HRNVS 的高質量結果。項目頁面：https://chchnii.github.io/GaussianSR/

English

Achieving high-resolution novel view synthesis (HRNVS) from low-resolution input views is a challenging task due to the lack of high-resolution data. Previous methods optimize high-resolution Neural Radiance Field (NeRF) from low-resolution input views but suffer from slow rendering speed. In this work, we base our method on 3D Gaussian Splatting (3DGS) due to its capability of producing high-quality images at a faster rendering speed. To alleviate the shortage of data for higher-resolution synthesis, we propose to leverage off-the-shelf 2D diffusion priors by distilling the 2D knowledge into 3D with Score Distillation Sampling (SDS). Nevertheless, applying SDS directly to Gaussian-based 3D super-resolution leads to undesirable and redundant 3D Gaussian primitives, due to the randomness brought by generative priors. To mitigate this issue, we introduce two simple yet effective techniques to reduce stochastic disturbances introduced by SDS. Specifically, we 1) shrink the range of diffusion timestep in SDS with an annealing strategy; 2) randomly discard redundant Gaussian primitives during densification. Extensive experiments have demonstrated that our proposed GaussainSR can attain high-quality results for HRNVS with only low-resolution inputs on both synthetic and real-world datasets. Project page: https://chchnii.github.io/GaussianSR/