GECO：生成式图像到三维在一秒内

GECO: Generative Image-to-3D within a SECOnd

May 30, 2024

作者: Chen Wang, Jiatao Gu, Xiaoxiao Long, Yuan Liu, Lingjie Liu

cs.AI

摘要

近年来，3D生成取得了显著进展。现有技术，如分数蒸馏方法，产生了显著的结果，但需要进行广泛的场景优化，影响了时间效率。另外，基于重建的方法优先考虑效率，但由于对不确定性的有限处理而牺牲了质量。我们引入了GECO，这是一种新颖的高质量3D生成建模方法，能在一秒内运行。我们的方法通过两阶段方法解决了当前方法中普遍存在的不确定性和低效率问题。在初始阶段，我们训练了一个单步多视角生成模型，并采用分数蒸馏。然后，对多视角预测中的视角不一致性挑战应用第二阶段蒸馏。这两阶段过程确保了对3D生成的平衡处理，优化了质量和效率。我们的全面实验表明，GECO实现了具有前所未有效率水平的高质量图像到3D生成。

English

3D generation has seen remarkable progress in recent years. Existing techniques, such as score distillation methods, produce notable results but require extensive per-scene optimization, impacting time efficiency. Alternatively, reconstruction-based approaches prioritize efficiency but compromise quality due to their limited handling of uncertainty. We introduce GECO, a novel method for high-quality 3D generative modeling that operates within a second. Our approach addresses the prevalent issues of uncertainty and inefficiency in current methods through a two-stage approach. In the initial stage, we train a single-step multi-view generative model with score distillation. Then, a second-stage distillation is applied to address the challenge of view inconsistency from the multi-view prediction. This two-stage process ensures a balanced approach to 3D generation, optimizing both quality and efficiency. Our comprehensive experiments demonstrate that GECO achieves high-quality image-to-3D generation with an unprecedented level of efficiency.