GECO：SECOnd 內的生成式圖像至 3D

GECO: Generative Image-to-3D within a SECOnd

May 30, 2024

作者: Chen Wang, Jiatao Gu, Xiaoxiao Long, Yuan Liu, Lingjie Liu

cs.AI

摘要

近年來，3D生成技術取得了顯著進展。現有的技術，如得分蒸餾方法，產生了顯著的結果，但需要進行大量的場景優化，影響了時間效率。相反，基於重建的方法優先考慮效率，但由於對不確定性的處理有限，會影響質量。我們介紹了GECO，一種新穎的高質量3D生成建模方法，操作時間僅需一秒。我們的方法通過兩階段方法解決了當前方法中普遍存在的不確定性和低效率問題。在初始階段，我們使用得分蒸餾訓練單步多視圖生成模型。然後，對多視圖預測中的視圖不一致性進行第二階段蒸餾。這個兩階段過程確保了對3D生成的平衡處理，優化了質量和效率。我們的全面實驗表明，GECO實現了具有前所未有效率水平的高質量圖像到3D生成。

English

3D generation has seen remarkable progress in recent years. Existing techniques, such as score distillation methods, produce notable results but require extensive per-scene optimization, impacting time efficiency. Alternatively, reconstruction-based approaches prioritize efficiency but compromise quality due to their limited handling of uncertainty. We introduce GECO, a novel method for high-quality 3D generative modeling that operates within a second. Our approach addresses the prevalent issues of uncertainty and inefficiency in current methods through a two-stage approach. In the initial stage, we train a single-step multi-view generative model with score distillation. Then, a second-stage distillation is applied to address the challenge of view inconsistency from the multi-view prediction. This two-stage process ensures a balanced approach to 3D generation, optimizing both quality and efficiency. Our comprehensive experiments demonstrate that GECO achieves high-quality image-to-3D generation with an unprecedented level of efficiency.