使用学习的流形校正进行分数蒸馏抽样

摘要

得分蒸馏采样（SDS）是一种最近广受欢迎的方法，它依赖于图像扩散模型来控制使用文本提示的优化问题。在本文中，我们对SDS损失函数进行了深入分析，识别了其公式中固有的问题，并提出了一个出乎意料但有效的修复方案。具体而言，我们将损失分解为不同因素，并分离出负责嘈杂梯度的组件。在原始公式中，高文本引导被用来解决噪声问题，导致了不良的副作用。相反，我们训练一个浅层网络，模仿图像扩散模型随时间变化的去噪不足，以有效地将其剔除。我们通过多个定性和定量实验展示了我们新颖损失公式的多功能性和有效性，包括基于优化的图像合成和编辑，零样本图像翻译网络训练，以及文本到3D合成。

English

Score Distillation Sampling (SDS) is a recent but already widely popular method that relies on an image diffusion model to control optimization problems using text prompts. In this paper, we conduct an in-depth analysis of the SDS loss function, identify an inherent problem with its formulation, and propose a surprisingly easy but effective fix. Specifically, we decompose the loss into different factors and isolate the component responsible for noisy gradients. In the original formulation, high text guidance is used to account for the noise, leading to unwanted side effects. Instead, we train a shallow network mimicking the timestep-dependent denoising deficiency of the image diffusion model in order to effectively factor it out. We demonstrate the versatility and the effectiveness of our novel loss formulation through several qualitative and quantitative experiments, including optimization-based image synthesis and editing, zero-shot image translation network training, and text-to-3D synthesis.

使用学习的流形校正进行分数蒸馏抽样

Score Distillation Sampling with Learned Manifold Corrective

摘要

Support