稳定的分数蒸馏用于高质量的3D生成

Stable Score Distillation for High-Quality 3D Generation

December 14, 2023

作者: Boshi Tang, Jianan Wang, Zhiyong Wu, Lei Zhang

cs.AI

摘要

得分蒸馏采样（Score Distillation Sampling，SDS）在条件3D内容生成方面表现出卓越性能。然而，对SDS公式的全面理解仍然不足，阻碍了3D生成的发展。在本研究中，我们将SDS解释为三个功能组件的组合：模式解耦、模式寻找和减少方差项，并分析每个组件的特性。我们展示了由于监督项固有缺陷导致的过度平滑和颜色饱和等问题，并揭示了SDS引入的减少方差项是次优的。此外，我们阐明了采用大型无分类器引导（Classifier-Free Guidance，CFG）尺度进行3D生成的原因。基于分析，我们提出了一种简单而有效的方法，称为稳定得分蒸馏（Stable Score Distillation，SSD），可以策略性地组织每个项以实现高质量的3D生成。大量实验证实了我们方法的有效性，展示了其能够生成高保真度的3D内容，即使在最具挑战性的NeRF表示条件下，也不会出现过度平滑和过度饱和等问题。

English

Score Distillation Sampling (SDS) has exhibited remarkable performance in conditional 3D content generation. However, a comprehensive understanding of the SDS formulation is still lacking, hindering the development of 3D generation. In this work, we present an interpretation of SDS as a combination of three functional components: mode-disengaging, mode-seeking and variance-reducing terms, and analyze the properties of each. We show that problems such as over-smoothness and color-saturation result from the intrinsic deficiency of the supervision terms and reveal that the variance-reducing term introduced by SDS is sub-optimal. Additionally, we shed light on the adoption of large Classifier-Free Guidance (CFG) scale for 3D generation. Based on the analysis, we propose a simple yet effective approach named Stable Score Distillation (SSD) which strategically orchestrates each term for high-quality 3D generation. Extensive experiments validate the efficacy of our approach, demonstrating its ability to generate high-fidelity 3D content without succumbing to issues such as over-smoothness and over-saturation, even under low CFG conditions with the most challenging NeRF representation.