使用分类器分数蒸馏的文本到3D转换

Text-to-3D with classifier score distillation

October 30, 2023

作者: Xin Yu, Yuan-Chen Guo, Yangguang Li, Ding Liang, Song-Hai Zhang, Xiaojuan Qi

cs.AI

摘要

最近，文本到3D生成取得了显著进展，特别是基于得分蒸馏采样（SDS）的方法，利用预训练的2D扩散模型。虽然无分类器指导的使用被广泛认为对于成功优化至关重要，但被视为辅助技巧而非最重要的组成部分。在本文中，我们重新评估了分类器无指导在得分蒸馏中的作用，并发现了一个令人惊讶的发现：仅凭指导就足以实现有效的文本到3D生成任务。我们将这种方法命名为分类器得分蒸馏（CSD），可以解释为在生成过程中使用隐式分类模型。这种新视角揭示了对于理解现有技术的新见解。我们验证了CSD在各种文本到3D任务中的有效性，包括形状生成、纹理合成和形状编辑，在结果上优于现有技术。我们的项目页面是https://xinyu-andy.github.io/Classifier-Score-Distillation

English

Text-to-3D generation has made remarkable progress recently, particularly with methods based on Score Distillation Sampling (SDS) that leverages pre-trained 2D diffusion models. While the usage of classifier-free guidance is well acknowledged to be crucial for successful optimization, it is considered an auxiliary trick rather than the most essential component. In this paper, we re-evaluate the role of classifier-free guidance in score distillation and discover a surprising finding: the guidance alone is enough for effective text-to-3D generation tasks. We name this method Classifier Score Distillation (CSD), which can be interpreted as using an implicit classification model for generation. This new perspective reveals new insights for understanding existing techniques. We validate the effectiveness of CSD across a variety of text-to-3D tasks including shape generation, texture synthesis, and shape editing, achieving results superior to those of state-of-the-art methods. Our project page is https://xinyu-andy.github.io/Classifier-Score-Distillation