MicroDreamer：基于评分的迭代重建，在20秒内实现零样本3D生成

MicroDreamer: Zero-shot 3D Generation in sim20 Seconds by Score-based Iterative Reconstruction

April 30, 2024

作者: Luxi Chen, Zhengyi Wang, Chongxuan Li, Tingting Gao, Hang Su, Jun Zhu

cs.AI

摘要

基于优化的方法，如分数蒸馏采样（SDS），在零样本3D生成中显示出潜力，但由于每个样本需要大量函数评估（NFEs），效率较低。在本文中，我们引入基于分数的迭代重建（SIR），这是一种高效且通用的用于3D生成的算法，采用多视角基于分数的扩散模型。给定扩散模型生成的图像，SIR通过反复优化3D参数来减少NFEs，与SDS中的单次优化不同，模拟3D重建过程。通过在像素空间中进行优化等其他改进，我们提出了一种称为MicroDreamer的高效方法，通常适用于各种3D表示和3D生成任务。特别是，在保持可比性能的同时，MicroDreamer在生成神经辐射场方面比SDS快5-20倍，并且在单个A100 GPU上从3D高斯分裂生成网格大约需要20秒，将最快的零样本基线DreamGaussian的时间减半。我们的代码可在https://github.com/ML-GSAI/MicroDreamer找到。

English

Optimization-based approaches, such as score distillation sampling (SDS), show promise in zero-shot 3D generation but suffer from low efficiency, primarily due to the high number of function evaluations (NFEs) required for each sample. In this paper, we introduce score-based iterative reconstruction (SIR), an efficient and general algorithm for 3D generation with a multi-view score-based diffusion model. Given the images produced by the diffusion model, SIR reduces NFEs by repeatedly optimizing 3D parameters, unlike the single optimization in SDS, mimicking the 3D reconstruction process. With other improvements including optimization in the pixel space, we present an efficient approach called MicroDreamer that generally applies to various 3D representations and 3D generation tasks. In particular, retaining a comparable performance, MicroDreamer is 5-20 times faster than SDS in generating neural radiance field and takes about 20 seconds to generate meshes from 3D Gaussian splitting on a single A100 GPU, halving the time of the fastest zero-shot baseline, DreamGaussian. Our code is available at https://github.com/ML-GSAI/MicroDreamer.