ATT3D：摊销文本到三维物体合成

ATT3D: Amortized Text-to-3D Object Synthesis

June 6, 2023

作者: Jonathan Lorraine, Kevin Xie, Xiaohui Zeng, Chen-Hsuan Lin, Towaki Takikawa, Nicholas Sharp, Tsung-Yi Lin, Ming-Yu Liu, Sanja Fidler, James Lucas

cs.AI

摘要

文本到三维建模通过将生成式文本到图像模型与图像到三维的方法（如神经辐射场）相结合，取得了令人振奋的进展。DreamFusion 最近取得了高质量的结果，但需要通过冗长的逐提示优化来创建三维对象。为解决这一问题，我们通过在统一模型上同时训练多个提示，而不是分开训练，来分期偿还文本提示的优化。通过这种方式，我们在整个提示集合上共享计算，在比逐提示优化更短的时间内进行训练。我们的框架 - 分期偿还文本到三维（ATT3D）- 可以实现提示之间的知识共享，以便泛化到未见过的设置，并在文本之间实现平滑插值，用于新颖资产和简单动画。

English

Text-to-3D modelling has seen exciting progress by combining generative text-to-image models with image-to-3D methods like Neural Radiance Fields. DreamFusion recently achieved high-quality results but requires a lengthy, per-prompt optimization to create 3D objects. To address this, we amortize optimization over text prompts by training on many prompts simultaneously with a unified model, instead of separately. With this, we share computation across a prompt set, training in less time than per-prompt optimization. Our framework - Amortized text-to-3D (ATT3D) - enables knowledge-sharing between prompts to generalize to unseen setups and smooth interpolations between text for novel assets and simple animations.