SteinDreamer：通过Stein恒等式实现文本到3D分数提炼的方差减少

摘要

评分蒸馏已成为文本到3D资产合成中最普遍的方法之一。本质上，评分蒸馏通过提升和反向传播在不同视角上平均得分来更新3D参数。在本文中，我们揭示评分蒸馏中的梯度估计固有地具有高方差。通过方差缩减的视角，SDS和VSD的有效性可以被解释为对蒸馏得分的蒙特卡洛估计器应用各种控制变量的应用。受此反思的启发，并基于Stein恒等式，我们提出了一种更一般的解决方案来减少评分蒸馏的方差，称为Stein评分蒸馏（SSD）。SSD结合了由Stein恒等式构建的控制变量，允许任意基线函数。这使我们能够包括灵活的引导先验和网络架构，以明确优化方差缩减。在我们的实验中，名为SteinDreamer的整体流程通过使用单眼深度估计器实例化控制变量来实现。结果表明，SSD能够有效减少蒸馏方差，并持续改善对象和场景级别生成的视觉质量。此外，我们证明SteinDreamer由于更稳定的梯度更新而实现比现有方法更快的收敛。

English

Score distillation has emerged as one of the most prevalent approaches for text-to-3D asset synthesis. Essentially, score distillation updates 3D parameters by lifting and back-propagating scores averaged over different views. In this paper, we reveal that the gradient estimation in score distillation is inherent to high variance. Through the lens of variance reduction, the effectiveness of SDS and VSD can be interpreted as applications of various control variates to the Monte Carlo estimator of the distilled score. Motivated by this rethinking and based on Stein's identity, we propose a more general solution to reduce variance for score distillation, termed Stein Score Distillation (SSD). SSD incorporates control variates constructed by Stein identity, allowing for arbitrary baseline functions. This enables us to include flexible guidance priors and network architectures to explicitly optimize for variance reduction. In our experiments, the overall pipeline, dubbed SteinDreamer, is implemented by instantiating the control variate with a monocular depth estimator. The results suggest that SSD can effectively reduce the distillation variance and consistently improve visual quality for both object- and scene-level generation. Moreover, we demonstrate that SteinDreamer achieves faster convergence than existing methods due to more stable gradient updates.

SteinDreamer：通过Stein恒等式实现文本到3D分数提炼的方差减少

SteinDreamer: Variance Reduction for Text-to-3D Score Distillation via Stein Identity

摘要

Support