FDGaussian：通過幾何感知擴散模型從單張圖像快速生成高斯樣本

FDGaussian: Fast Gaussian Splatting from Single Image via Geometric-aware Diffusion Model

March 15, 2024

作者: Qijun Feng, Zhen Xing, Zuxuan Wu, Yu-Gang Jiang

cs.AI

摘要

從單視圖圖像重建詳細的3D物體仍然是一項具有挑戰性的任務，這是因為可用信息有限。在本文中，我們介紹了FDGaussian，這是一種新穎的單圖像3D重建的兩階段框架。最近的方法通常利用預先訓練的2D擴散模型從輸入圖像生成合理的新視圖，但它們遇到多視圖不一致或幾何保真度不足的問題。為了克服這些挑戰，我們提出了一種正交平面分解機制，從2D輸入中提取3D幾何特徵，從而實現一致的多視圖圖像生成。此外，我們進一步加速了最先進的高斯濺射，並引入了對焦注意力機制，以融合來自不同視角的圖像。我們展示了FDGaussian生成的圖像在不同視角之間具有高一致性，並在質量和量化方面重建高質量的3D物體。更多範例可在我們的網站https://qjfeng.net/FDGaussian/找到。

English

Reconstructing detailed 3D objects from single-view images remains a challenging task due to the limited information available. In this paper, we introduce FDGaussian, a novel two-stage framework for single-image 3D reconstruction. Recent methods typically utilize pre-trained 2D diffusion models to generate plausible novel views from the input image, yet they encounter issues with either multi-view inconsistency or lack of geometric fidelity. To overcome these challenges, we propose an orthogonal plane decomposition mechanism to extract 3D geometric features from the 2D input, enabling the generation of consistent multi-view images. Moreover, we further accelerate the state-of-the-art Gaussian Splatting incorporating epipolar attention to fuse images from different viewpoints. We demonstrate that FDGaussian generates images with high consistency across different views and reconstructs high-quality 3D objects, both qualitatively and quantitatively. More examples can be found at our website https://qjfeng.net/FDGaussian/.