PixSDS：為何潛在SDS會產生雜訊像素

摘要

分數蒸餾採樣（SDS）透過使用預訓練的擴散先驗最佳化渲染影像來實現文字轉 3D 生成，但潛在 SDS（latent SDS）常產生結構性色彩偽影與高頻紋理雜訊。我們識別出由 VAE 引起的像素漂移所導致的潛在 SDS 失敗模式：最佳化後的影像可能沿著 VAE 編碼器約束較弱的像素空間方向移動，因此其潛在表徵保持乾淨且具有語意意義，而影像本身卻累積可見的偽影。我們透過受控的 2D SDS 實驗、僅 VAE 的最佳化，以及一項簡化分析來支持此診斷，該分析顯示當到像素的逆映射約束不足時，類似編碼器的潛在目標會放大影像空間的雜訊。受此觀察啟發，我們提出 PixSDS，一種輕量級且與 VAE 一致的梯度修復方法。PixSDS 解碼潛在 SDS 的前瞻步驟，並使用解碼後的影像作為像素空間最佳化的乾淨方向，減少在與 VAE 不一致方向上的移動，而無需重新訓練擴散模型、更換渲染器或取代 SDS 目標。在 2D 最佳化與文字轉 3D 生成中的實驗顯示，PixSDS 能大幅減少結構性偽影，同時保留語意內容。程式碼已公開於 https://sevashasla.github.io/pixsds-webpage/。

English

Score Distillation Sampling (SDS) enables text-to-3D generation by optimizing rendered images with a pretrained diffusion prior, but latent SDS often produces structured color artifacts and high-frequency texture noise. We identify a failure mode of latent SDS caused by VAE-induced pixel drift: the optimized image can move along pixel-space directions that are weakly constrained by the VAE encoder, so its latent representation remains clean and semantically meaningful while the image itself accumulates visible artifacts. We support this diagnosis with controlled 2D SDS experiments, VAE-only optimization, and a simplified analysis showing that encoder-like latent objectives can amplify image-space noise when the inverse mapping to pixels is underconstrained. Motivated by this observation, we propose PixSDS, a lightweight VAE-consistent gradient repair method. PixSDS decodes a latent SDS lookahead step and uses the decoded image as a clean direction for pixel-space optimization, reducing motion in VAE-inconsistent directions without retraining the diffusion model, changing the renderer, or replacing the SDS objective. Experiments in 2D optimization and text-to-3D generation show that PixSDS substantially reduces structured artifacts while preserving semantic content. Code is publicly available at https://sevashasla.github.io/pixsds-webpage/.