GSFixer：基于参考引导视频扩散先验的3D高斯溅射优化

GSFixer: Improving 3D Gaussian Splatting with Reference-Guided Video Diffusion Priors

August 13, 2025

作者: Xingyilang Yin, Qi Zhang, Jiahao Chang, Ying Feng, Qingnan Fan, Xi Yang, Chi-Man Pun, Huaqi Zhang, Xiaodong Cun

cs.AI

摘要

利用3D高斯泼溅（3DGS）从稀疏视角重建3D场景，由于信息不足，往往导致明显伪影，是一个不适定问题。尽管近期方法尝试利用生成先验来补全欠约束区域的信息，但在生成与输入观测保持一致的内容方面仍面临挑战。为解决这一难题，我们提出了GSFixer，一个旨在提升从稀疏输入重建的3DGS表示质量的新颖框架。我们方法的核心在于基于DiT的视频扩散模型构建的参考引导视频修复模型，该模型在成对的含伪影3DGS渲染图与干净帧上训练，并附加了基于参考的条件。将输入稀疏视图视为参考，我们的模型整合了从视觉几何基础模型中提取的参考视图的2D语义特征和3D几何特征，增强了在修复含伪影新视图时的语义连贯性和3D一致性。此外，考虑到缺乏适合评估3DGS伪影修复的基准，我们提出了DL3DV-Res，其中包含使用低质量3DGS渲染的伪影帧。大量实验证明，我们的GSFixer在3DGS伪影修复和稀疏视图3D重建方面超越了当前最先进的方法。项目页面：https://github.com/GVCLab/GSFixer。

English

Reconstructing 3D scenes using 3D Gaussian Splatting (3DGS) from sparse views is an ill-posed problem due to insufficient information, often resulting in noticeable artifacts. While recent approaches have sought to leverage generative priors to complete information for under-constrained regions, they struggle to generate content that remains consistent with input observations. To address this challenge, we propose GSFixer, a novel framework designed to improve the quality of 3DGS representations reconstructed from sparse inputs. The core of our approach is the reference-guided video restoration model, built upon a DiT-based video diffusion model trained on paired artifact 3DGS renders and clean frames with additional reference-based conditions. Considering the input sparse views as references, our model integrates both 2D semantic features and 3D geometric features of reference views extracted from the visual geometry foundation model, enhancing the semantic coherence and 3D consistency when fixing artifact novel views. Furthermore, considering the lack of suitable benchmarks for 3DGS artifact restoration evaluation, we present DL3DV-Res which contains artifact frames rendered using low-quality 3DGS. Extensive experiments demonstrate our GSFixer outperforms current state-of-the-art methods in 3DGS artifact restoration and sparse-view 3D reconstruction. Project page: https://github.com/GVCLab/GSFixer.