Wonder3D：使用跨域扩散实现从单图像到3D

摘要

在这项工作中，我们介绍了Wonder3D，一种从单视图图像高效生成高保真纹理网格的新方法。最近基于得分蒸馏采样（SDS）的方法已经显示出从2D扩散先验中恢复3D几何的潜力，但它们通常受到耗时的每个形状优化和不一致几何的困扰。相比之下，某些工作通过快速网络推断直接生成3D信息，但它们的结果通常质量较低且缺乏几何细节。为了全面提高图像到3D任务的质量、一致性和效率，我们提出了一个跨领域扩散模型，生成多视角法线图和相应的彩色图像。为了确保一致性，我们采用了多视角跨领域注意机制，促进视图和模态之间的信息交换。最后，我们介绍了一种几何感知法线融合算法，从多视角的2D表示中提取高质量的表面。我们的广泛评估表明，与先前的工作相比，我们的方法实现了高质量的重建结果、稳健的泛化能力和相当不错的效率。

English

In this work, we introduce Wonder3D, a novel method for efficiently generating high-fidelity textured meshes from single-view images.Recent methods based on Score Distillation Sampling (SDS) have shown the potential to recover 3D geometry from 2D diffusion priors, but they typically suffer from time-consuming per-shape optimization and inconsistent geometry. In contrast, certain works directly produce 3D information via fast network inferences, but their results are often of low quality and lack geometric details. To holistically improve the quality, consistency, and efficiency of image-to-3D tasks, we propose a cross-domain diffusion model that generates multi-view normal maps and the corresponding color images. To ensure consistency, we employ a multi-view cross-domain attention mechanism that facilitates information exchange across views and modalities. Lastly, we introduce a geometry-aware normal fusion algorithm that extracts high-quality surfaces from the multi-view 2D representations. Our extensive evaluations demonstrate that our method achieves high-quality reconstruction results, robust generalization, and reasonably good efficiency compared to prior works.

Wonder3D：使用跨域扩散实现从单图像到3D

Wonder3D: Single Image to 3D using Cross-Domain Diffusion

摘要

Support