MicroDreamer：基於分數的迭代重建，在20秒內實現零樣本3D生成

MicroDreamer: Zero-shot 3D Generation in sim20 Seconds by Score-based Iterative Reconstruction

April 30, 2024

作者: Luxi Chen, Zhengyi Wang, Chongxuan Li, Tingting Gao, Hang Su, Jun Zhu

cs.AI

摘要

基於優化的方法，如分數蒸餾取樣（SDS），在零樣本3D生成方面表現出潛力，但由於每個樣本所需的函數評估次數（NFEs）較高，效率較低。本文介紹了基於分數的迭代重建（SIR），這是一種高效且通用的用於3D生成的算法，採用多視圖基於分數的擴散模型。給定擴散模型生成的圖像，SIR通過反复優化3D參數來減少NFEs，與SDS中的單次優化不同，模擬了3D重建過程。通過在像素空間中進行優化等其他改進，我們提出了一種高效的方法，稱為MicroDreamer，通常適用於各種3D表示和3D生成任務。特別是，在保持可比性能的情況下，MicroDreamer在生成神經輻射場方面比SDS快5-20倍，並且在單個A100 GPU上從3D高斯分割生成網格大約需要20秒，時間是最快零樣本基線DreamGaussian的一半。我們的代碼可在https://github.com/ML-GSAI/MicroDreamer找到。

English

Optimization-based approaches, such as score distillation sampling (SDS), show promise in zero-shot 3D generation but suffer from low efficiency, primarily due to the high number of function evaluations (NFEs) required for each sample. In this paper, we introduce score-based iterative reconstruction (SIR), an efficient and general algorithm for 3D generation with a multi-view score-based diffusion model. Given the images produced by the diffusion model, SIR reduces NFEs by repeatedly optimizing 3D parameters, unlike the single optimization in SDS, mimicking the 3D reconstruction process. With other improvements including optimization in the pixel space, we present an efficient approach called MicroDreamer that generally applies to various 3D representations and 3D generation tasks. In particular, retaining a comparable performance, MicroDreamer is 5-20 times faster than SDS in generating neural radiance field and takes about 20 seconds to generate meshes from 3D Gaussian splitting on a single A100 GPU, halving the time of the fastest zero-shot baseline, DreamGaussian. Our code is available at https://github.com/ML-GSAI/MicroDreamer.