PixSDS：为何潜在SDS会产生噪声像素

摘要

分数蒸馏采样（Score Distillation Sampling, SDS）通过利用预训练的扩散先验优化渲染图像来实现文本到3D生成，但潜在SDS常常产生结构性的颜色伪影和高频纹理噪声。我们识别出潜在SDS的一种失效模式，其由VAE引发的像素漂移所导致：优化图像可能沿着像素空间中受VAE编码器约束较弱的方向移动，因此其潜在表示保持干净且语义有意义，而图像本身则累积可见伪影。我们通过受控的2D SDS实验、仅VAE优化和一项简化分析来支持这一诊断，该分析表明，当到像素的逆映射约束不足时，类编码器的潜在目标会放大图像空间中的噪声。基于这一观察，我们提出PixSDS——一种轻量级的VAE一致梯度修复方法。PixSDS对潜在SDS的前瞻步骤进行解码，并将解码后的图像用作像素空间优化的干净方向，从而无需重新训练扩散模型、更改渲染器或替换SDS目标，即可减少在VAE不一致方向上的移动。在2D优化和文本到3D生成上的实验表明，PixSDS在保留语义内容的同时显著减少了结构性伪影。代码可在https://sevashasla.github.io/pixsds-webpage/ 获取。

English

Score Distillation Sampling (SDS) enables text-to-3D generation by optimizing rendered images with a pretrained diffusion prior, but latent SDS often produces structured color artifacts and high-frequency texture noise. We identify a failure mode of latent SDS caused by VAE-induced pixel drift: the optimized image can move along pixel-space directions that are weakly constrained by the VAE encoder, so its latent representation remains clean and semantically meaningful while the image itself accumulates visible artifacts. We support this diagnosis with controlled 2D SDS experiments, VAE-only optimization, and a simplified analysis showing that encoder-like latent objectives can amplify image-space noise when the inverse mapping to pixels is underconstrained. Motivated by this observation, we propose PixSDS, a lightweight VAE-consistent gradient repair method. PixSDS decodes a latent SDS lookahead step and uses the decoded image as a clean direction for pixel-space optimization, reducing motion in VAE-inconsistent directions without retraining the diffusion model, changing the renderer, or replacing the SDS objective. Experiments in 2D optimization and text-to-3D generation show that PixSDS substantially reduces structured artifacts while preserving semantic content. Code is publicly available at https://sevashasla.github.io/pixsds-webpage/.