FSGS：使用高斯飞溅实现实时小样本视图合成

摘要

从有限观测中合成新视角仍然是一个重要且持久的任务。然而，现有基于 NeRF 的少样本视角合成方法往往在追求准确的 3D 表示时牺牲了高效性。为了解决这一挑战，我们提出了一种基于 3D 高斯光斑的少样本视角合成框架，可以实现实时且照片逼真的视角合成，仅需三个训练视角。所提出的方法，命名为 FSGS，通过精心设计的高斯反卷积过程处理极其稀疏的初始化 SfM 点。我们的方法通过迭代在最具代表性的位置周围分布新的高斯函数，随后填充空白区域中的局部细节。我们还在高斯优化过程中集成了大规模预训练的单目深度估计器，利用在线增强视角指导几何优化朝向最佳解决方案。从有限输入视点观察到的稀疏点开始，我们的 FSGS 能够准确扩展到未见区域，全面覆盖场景并提升新视角的渲染质量。总体而言，FSGS 在准确性和渲染效率上在多个数据集（包括 LLFF、Mip-NeRF360 和 Blender）中均取得了最先进的性能。项目网站：https://zehaozhu.github.io/FSGS/.

English

Novel view synthesis from limited observations remains an important and persistent task. However, high efficiency in existing NeRF-based few-shot view synthesis is often compromised to obtain an accurate 3D representation. To address this challenge, we propose a few-shot view synthesis framework based on 3D Gaussian Splatting that enables real-time and photo-realistic view synthesis with as few as three training views. The proposed method, dubbed FSGS, handles the extremely sparse initialized SfM points with a thoughtfully designed Gaussian Unpooling process. Our method iteratively distributes new Gaussians around the most representative locations, subsequently infilling local details in vacant areas. We also integrate a large-scale pre-trained monocular depth estimator within the Gaussians optimization process, leveraging online augmented views to guide the geometric optimization towards an optimal solution. Starting from sparse points observed from limited input viewpoints, our FSGS can accurately grow into unseen regions, comprehensively covering the scene and boosting the rendering quality of novel views. Overall, FSGS achieves state-of-the-art performance in both accuracy and rendering efficiency across diverse datasets, including LLFF, Mip-NeRF360, and Blender. Project website: https://zehaozhu.github.io/FSGS/.

FSGS：使用高斯飞溅实现实时小样本视图合成

FSGS: Real-Time Few-shot View Synthesis using Gaussian Splatting

摘要

Support