Wonder3D：使用跨領域擴散從單張圖像生成3D

摘要

在這份工作中，我們介紹了 Wonder3D，一種從單視角圖像高效生成高保真度紋理網格的新方法。最近基於得分蒸餾取樣（SDS）的方法已經展示了從2D擴散先驗中恢復3D幾何的潛力，但通常受到耗時的每形狀優化和不一致的幾何的困擾。相比之下，某些作品通過快速網絡推斷直接生成3D信息，但其結果通常質量低且缺乏幾何細節。為了全面提高圖像到3D任務的質量、一致性和效率，我們提出了一種跨領域擴散模型，生成多視角法線圖和相應的彩色圖像。為確保一致性，我們採用了多視角跨領域注意機制，促進跨視圖和模態之間的信息交換。最後，我們介紹了一種幾何感知法線融合算法，從多視角2D表示中提取高質量表面。我們的廣泛評估表明，與先前作品相比，我們的方法實現了高質量的重建結果、強大的泛化能力和相當不錯的效率。

English

In this work, we introduce Wonder3D, a novel method for efficiently generating high-fidelity textured meshes from single-view images.Recent methods based on Score Distillation Sampling (SDS) have shown the potential to recover 3D geometry from 2D diffusion priors, but they typically suffer from time-consuming per-shape optimization and inconsistent geometry. In contrast, certain works directly produce 3D information via fast network inferences, but their results are often of low quality and lack geometric details. To holistically improve the quality, consistency, and efficiency of image-to-3D tasks, we propose a cross-domain diffusion model that generates multi-view normal maps and the corresponding color images. To ensure consistency, we employ a multi-view cross-domain attention mechanism that facilitates information exchange across views and modalities. Lastly, we introduce a geometry-aware normal fusion algorithm that extracts high-quality surfaces from the multi-view 2D representations. Our extensive evaluations demonstrate that our method achieves high-quality reconstruction results, robust generalization, and reasonably good efficiency compared to prior works.

Wonder3D：使用跨領域擴散從單張圖像生成3D

Wonder3D: Single Image to 3D using Cross-Domain Diffusion

摘要

Support