GSFixer：利用參考引導的視頻擴散先驗改進3D高斯潑濺技術

GSFixer: Improving 3D Gaussian Splatting with Reference-Guided Video Diffusion Priors

August 13, 2025

作者: Xingyilang Yin, Qi Zhang, Jiahao Chang, Ying Feng, Qingnan Fan, Xi Yang, Chi-Man Pun, Huaqi Zhang, Xiaodong Cun

cs.AI

摘要

基於稀疏視角重建三維場景的三維高斯潑濺（3DGS）技術，由於信息不足，往往會產生明顯的偽影，這是一個病態問題。儘管近期研究嘗試利用生成先驗來補全約束不足區域的信息，但這些方法在生成與輸入觀測保持一致的內容方面仍面臨挑戰。為解決這一難題，我們提出了GSFixer，這是一個旨在提升從稀疏輸入重建的3DGS表示質量的新穎框架。我們方法的核心是基於參考引導的視頻修復模型，該模型建立在一個DiT基礎的視頻擴散模型之上，該模型在配對的偽影3DGS渲染圖與乾淨幀以及附加的基於參考的條件下進行訓練。將輸入的稀疏視角視為參考，我們的模型整合了從視覺幾何基礎模型中提取的參考視角的二維語義特徵和三維幾何特徵，從而增強了在修復偽影新視角時的語義連貫性和三維一致性。此外，考慮到缺乏適合評估3DGS偽影修復的基準，我們提出了DL3DV-Res，其中包含了使用低質量3DGS渲染的偽影幀。大量實驗證明，我們的GSFixer在3DGS偽影修復和稀疏視角三維重建方面超越了當前最先進的方法。項目頁面：https://github.com/GVCLab/GSFixer。

English

Reconstructing 3D scenes using 3D Gaussian Splatting (3DGS) from sparse views is an ill-posed problem due to insufficient information, often resulting in noticeable artifacts. While recent approaches have sought to leverage generative priors to complete information for under-constrained regions, they struggle to generate content that remains consistent with input observations. To address this challenge, we propose GSFixer, a novel framework designed to improve the quality of 3DGS representations reconstructed from sparse inputs. The core of our approach is the reference-guided video restoration model, built upon a DiT-based video diffusion model trained on paired artifact 3DGS renders and clean frames with additional reference-based conditions. Considering the input sparse views as references, our model integrates both 2D semantic features and 3D geometric features of reference views extracted from the visual geometry foundation model, enhancing the semantic coherence and 3D consistency when fixing artifact novel views. Furthermore, considering the lack of suitable benchmarks for 3DGS artifact restoration evaluation, we present DL3DV-Res which contains artifact frames rendered using low-quality 3DGS. Extensive experiments demonstrate our GSFixer outperforms current state-of-the-art methods in 3DGS artifact restoration and sparse-view 3D reconstruction. Project page: https://github.com/GVCLab/GSFixer.