使用分類器分數蒸餾的文本轉3D

Text-to-3D with classifier score distillation

October 30, 2023

作者: Xin Yu, Yuan-Chen Guo, Yangguang Li, Ding Liang, Song-Hai Zhang, Xiaojuan Qi

cs.AI

摘要

最近，文本轉3D生成取得了顯著進展，特別是基於得分蒸餾取樣（SDS）的方法，利用預先訓練的2D擴散模型。儘管無分類器指導的使用被廣泛認為對於成功優化至關重要，但被視為輔助技巧而非最重要的組成部分。在本文中，我們重新評估了無分類器指導在得分蒸餾中的作用，並發現了一個令人驚訝的發現：單獨的指導足以進行有效的文本轉3D生成任務。我們將此方法命名為分類器得分蒸餾（CSD），可以解釋為使用隱式分類模型進行生成。這種新觀點揭示了對現有技術的新見解。我們驗證了CSD在各種文本轉3D任務中的有效性，包括形狀生成、紋理合成和形狀編輯，在這些任務中取得了優於最先進方法的結果。我們的項目頁面是https://xinyu-andy.github.io/Classifier-Score-Distillation

English

Text-to-3D generation has made remarkable progress recently, particularly with methods based on Score Distillation Sampling (SDS) that leverages pre-trained 2D diffusion models. While the usage of classifier-free guidance is well acknowledged to be crucial for successful optimization, it is considered an auxiliary trick rather than the most essential component. In this paper, we re-evaluate the role of classifier-free guidance in score distillation and discover a surprising finding: the guidance alone is enough for effective text-to-3D generation tasks. We name this method Classifier Score Distillation (CSD), which can be interpreted as using an implicit classification model for generation. This new perspective reveals new insights for understanding existing techniques. We validate the effectiveness of CSD across a variety of text-to-3D tasks including shape generation, texture synthesis, and shape editing, achieving results superior to those of state-of-the-art methods. Our project page is https://xinyu-andy.github.io/Classifier-Score-Distillation