ATT3D：攤銷式文本轉3D物體合成

ATT3D: Amortized Text-to-3D Object Synthesis

June 6, 2023

作者: Jonathan Lorraine, Kevin Xie, Xiaohui Zeng, Chen-Hsuan Lin, Towaki Takikawa, Nicholas Sharp, Tsung-Yi Lin, Ming-Yu Liu, Sanja Fidler, James Lucas

cs.AI

摘要

將文字轉換為3D建模已經取得了令人振奮的進展，透過將生成式文字轉圖像模型與圖像轉3D方法（如神經輻射場）相結合。DreamFusion最近取得了高質量的結果，但需要進行冗長的按提示優化才能創建3D物件。為了解決這個問題，我們通過使用統一模型，而不是分開訓練，將優化分攤到多個文本提示上。透過這種方式，我們在比逐個提示優化更短的時間內共享計算。我們的框架 - 分攤式文字轉3D（ATT3D） - 使提示之間的知識共享得以實現，以泛化到未見過的設置，並實現文本之間的平滑插值，用於新穎資產和簡單動畫。

English

Text-to-3D modelling has seen exciting progress by combining generative text-to-image models with image-to-3D methods like Neural Radiance Fields. DreamFusion recently achieved high-quality results but requires a lengthy, per-prompt optimization to create 3D objects. To address this, we amortize optimization over text prompts by training on many prompts simultaneously with a unified model, instead of separately. With this, we share computation across a prompt set, training in less time than per-prompt optimization. Our framework - Amortized text-to-3D (ATT3D) - enables knowledge-sharing between prompts to generalize to unseen setups and smooth interpolations between text for novel assets and simple animations.