DreamPolish：利用渐进几何生成进行域分数提炼

DreamPolish: Domain Score Distillation With Progressive Geometry Generation

November 3, 2024

作者: Yean Cheng, Ziqi Cai, Ming Ding, Wendi Zheng, Shiyu Huang, Yuxiao Dong, Jie Tang, Boxin Shi

cs.AI

摘要

我们介绍了DreamPolish，这是一个在生成精细几何和高质量纹理方面表现出色的文本到3D生成模型。在几何构建阶段，我们的方法利用多个神经表示来增强合成过程的稳定性。我们不仅仅依赖于新颖采样视图中的视图条件扩散先验，因为这经常会导致几何表面上不希望出现的伪影，我们还结合了额外的法线估计器来优化几何细节，这些细节是根据不同视角的视场来确定的。我们建议增加一个表面优化阶段，只需进行少量训练步骤，就可以有效地改进由于前几个阶段受到的有限指导而产生的伪影，并产生具有更理想几何的3D物体。在使用预训练文本到图像模型进行纹理生成的关键问题是在这些模型的广阔潜在分布中找到一个包含照片级和一致渲染的合适领域。在纹理生成阶段，我们引入了一种新颖的分数蒸馏目标，即域分数蒸馏（DSD），以引导神经表示朝向这样一个领域。我们从文本条件图像生成任务中的无分类器指导（CFG）中汲取灵感，并展示CFG和变分分布指导代表了梯度指导中的不同方面，对于提高纹理质量来说，这两个领域都是至关重要的。大量实验证明我们提出的模型可以生成具有优化表面和照片级纹理的3D资产，优于现有的最先进方法。

English

We introduce DreamPolish, a text-to-3D generation model that excels in producing refined geometry and high-quality textures. In the geometry construction phase, our approach leverages multiple neural representations to enhance the stability of the synthesis process. Instead of relying solely on a view-conditioned diffusion prior in the novel sampled views, which often leads to undesired artifacts in the geometric surface, we incorporate an additional normal estimator to polish the geometry details, conditioned on viewpoints with varying field-of-views. We propose to add a surface polishing stage with only a few training steps, which can effectively refine the artifacts attributed to limited guidance from previous stages and produce 3D objects with more desirable geometry. The key topic of texture generation using pretrained text-to-image models is to find a suitable domain in the vast latent distribution of these models that contains photorealistic and consistent renderings. In the texture generation phase, we introduce a novel score distillation objective, namely domain score distillation (DSD), to guide neural representations toward such a domain. We draw inspiration from the classifier-free guidance (CFG) in textconditioned image generation tasks and show that CFG and variational distribution guidance represent distinct aspects in gradient guidance and are both imperative domains for the enhancement of texture quality. Extensive experiments show our proposed model can produce 3D assets with polished surfaces and photorealistic textures, outperforming existing state-of-the-art methods.