PixRestore：基於像素擴散Transformer的統一影像修復

摘要

統一影像修復（UIR）旨在使用單一模型，從具有不同退化的低品質（LQ）影像中恢復高品質（HQ）內容。近期多數方法採用大型預訓練的文字轉影像（T2I）潛在擴散模型，因其具有強大的容量與生成先驗。然而，潛在T2I模型中的變分自編碼器（VAE）可能丟棄對修復敏感的細節，而開放式生成先驗則可能引入與內容不一致的偽影。我們提出PixRestore，一種用於UIR的無VAE像素空間擴散Transformer（DiT），其擴散骨幹完全從零開始訓練，不依賴T2I預訓練。PixRestore直接在分塊化像素上執行流匹配，在保持令牌序列可處理的同時保留細粒度細節。為了適應不同的退化，PixRestore學習利用LQ-HQ DINO特徵相似度來預測層特徵的可靠性。來自較可靠層的特徵被融合作為密集條件，而較不可靠的層則接受更強的高品質特徵監督，以促進退化移除。我們在大規模多場景、多退化的語料庫上訓練PixRestore，並進一步使用基於DINO的對抗目標將其微調為單步生成器，以實現高效推理。在公開基準與真實世界測試集上的實驗顯示，僅約50M參數且單步推理，PixRestore在競爭的UIR模型中達到了最佳的整體保真度、感知品質與對退化的穩健性，同時效率遠高於其他模型。更大的PixRestore變體可進一步提升效能，證明我們像素空間設計的可擴展性。程式碼與精選基準可在 https://github.com/csslc/PixRestore 取得。

English

Unified image restoration (UIR) aims to recover high-quality (HQ) content from low-quality (LQ) images with different degradations using a single model. Most recent methods adapt large pretrained text-to-image (T2I) latent diffusion models for their strong capacity and generative priors. However, the variational autoencoder (VAE) in latent T2I models may discard restoration-sensitive details, while the open-ended synthesis prior can introduce content-inconsistent artifacts. We present PixRestore, a VAE-free pixel-space Diffusion Transformer (DiT) for UIR, where the diffusion backbone is trained entirely from scratch, without relying on T2I pretraining. PixRestore performs flow matching directly on patchified pixels, preserving fine-grained details while keeping the token sequence tractable. To adapt to different degradations, PixRestore learns to predict the reliability of layer features using LQ--HQ DINO feature similarity. Features from more reliable layers are fused as dense conditioning, while less reliable layers receive stronger HQ-feature supervision to encourage degradation removal. We train PixRestore on a large-scale corpus of diverse scenes and degradations, and further finetune it into a one-step generator using DINO-based adversarial objectives for efficient inference. Experiments on public benchmarks and real-world test sets show that, with only about 50M parameters and single-step inference, PixRestore achieves the best overall fidelity, perceptual quality, and robustness to degradations among competing UIR models while being far more efficient. Larger PixRestore variants can further boost performance, demonstrating the scalability of our pixel-space design. Code and the curated benchmark can be found at https://github.com/csslc/PixRestore.