Vista3D：揭开单张图像的3D黑暗面

Vista3D: Unravel the 3D Darkside of a Single Image

September 18, 2024

作者: Qiuhong Shen, Xingyi Yang, Michael Bi Mi, Xinchao Wang

cs.AI

摘要

我们踏上古老的探索之旅：从仅仅一瞥可见部分揭示物体的隐藏维度。为了解决这一问题，我们提出了Vista3D，这是一个能够在短短5分钟内快速且一致地生成3D图像的框架。Vista3D的核心是一个两阶段方法：粗略阶段和精细阶段。在粗略阶段，我们通过从单个图像快速生成初始几何结构，采用高斯散点法。在精细阶段，我们直接从学习到的高斯散点法中提取有符号距离函数（SDF），并通过可微的等值面表示进行优化。此外，它通过使用两个独立的隐式函数来捕捉物体的可见和隐藏部分，提升了生成质量。此外，它通过角度扩散先验组合将2D扩散先验的梯度与3D感知扩散先验进行协调。通过广泛的评估，我们展示了Vista3D有效地在生成的3D物体之间保持了一致性和多样性的平衡。演示和代码将在https://github.com/florinshen/Vista3D 上提供。

English

We embark on the age-old quest: unveiling the hidden dimensions of objects from mere glimpses of their visible parts. To address this, we present Vista3D, a framework that realizes swift and consistent 3D generation within a mere 5 minutes. At the heart of Vista3D lies a two-phase approach: the coarse phase and the fine phase. In the coarse phase, we rapidly generate initial geometry with Gaussian Splatting from a single image. In the fine phase, we extract a Signed Distance Function (SDF) directly from learned Gaussian Splatting, optimizing it with a differentiable isosurface representation. Furthermore, it elevates the quality of generation by using a disentangled representation with two independent implicit functions to capture both visible and obscured aspects of objects. Additionally, it harmonizes gradients from 2D diffusion prior with 3D-aware diffusion priors by angular diffusion prior composition. Through extensive evaluation, we demonstrate that Vista3D effectively sustains a balance between the consistency and diversity of the generated 3D objects. Demos and code will be available at https://github.com/florinshen/Vista3D.