FDGaussian：通过几何感知扩散模型从单个图像快速生成高斯飞溅

FDGaussian: Fast Gaussian Splatting from Single Image via Geometric-aware Diffusion Model

March 15, 2024

作者: Qijun Feng, Zhen Xing, Zuxuan Wu, Yu-Gang Jiang

cs.AI

摘要

从单视图图像中重建详细的3D对象仍然是一项具有挑战性的任务，因为可用信息有限。在本文中，我们介绍了一种名为FDGaussian的新颖的单图像3D重建的两阶段框架。最近的方法通常利用预训练的2D扩散模型从输入图像生成可能的新视图，但它们在多视图不一致或几何保真度不足方面遇到问题。为了克服这些挑战，我们提出了一种正交平面分解机制，从2D输入中提取3D几何特征，实现了一致的多视图图像生成。此外，我们进一步加速了最先进的高斯光斑技术，结合了极线注意力来融合来自不同视角的图像。我们展示了FDGaussian能够生成在不同视角下具有高一致性的图像，并在定性和定量上重建高质量的3D对象。更多示例请访问我们的网站https://qjfeng.net/FDGaussian/。

English

Reconstructing detailed 3D objects from single-view images remains a challenging task due to the limited information available. In this paper, we introduce FDGaussian, a novel two-stage framework for single-image 3D reconstruction. Recent methods typically utilize pre-trained 2D diffusion models to generate plausible novel views from the input image, yet they encounter issues with either multi-view inconsistency or lack of geometric fidelity. To overcome these challenges, we propose an orthogonal plane decomposition mechanism to extract 3D geometric features from the 2D input, enabling the generation of consistent multi-view images. Moreover, we further accelerate the state-of-the-art Gaussian Splatting incorporating epipolar attention to fuse images from different viewpoints. We demonstrate that FDGaussian generates images with high consistency across different views and reconstructs high-quality 3D objects, both qualitatively and quantitatively. More examples can be found at our website https://qjfeng.net/FDGaussian/.