去噪作為適應：噪聲空間領域適應用於圖像修復

Denoising as Adaptation: Noise-Space Domain Adaptation for Image Restoration

June 26, 2024

作者: Kang Liao, Zongsheng Yue, Zhouxia Wang, Chen Change Loy

cs.AI

摘要

儘管基於學習的圖像修復方法取得了顯著進展，但由於在合成數據上進行訓練導致的實際場景的顯著領域差異，它們仍然在實際應用中面臨著有限的泛化困難。現有方法通過改進數據合成流程、估計降解核、應用深度內部學習以及執行領域適應和正則化來解決這個問題。先前的領域適應方法試圖通過在特徵空間或像素空間中學習領域不變的知識來彌合領域差距。然而，這些技術通常難以在穩定而緊湊的框架內擴展到低級別視覺任務。本文展示了通過噪聲空間使用擴散模型進行領域適應是可能的。具體來說，通過利用輔助條件輸入如何影響多步去噪過程的獨特特性，我們推導出一個有意義的擴散損失，該損失引導修復模型逐步將修復的合成和實際輸出與目標乾淨分佈對齊。我們稱這種方法為去噪適應。為了在聯合訓練期間防止捷徑，我們提出了重要策略，例如通道混洗層和擴散模型中的殘差交換對比學習。它們隱式地模糊了條件合成和實際數據之間的界限，並防止模型依賴容易識別的特徵。在三個經典圖像修復任務，即去噪、去模糊和去雨水，的實驗結果證明了所提方法的有效性。

English

Although learning-based image restoration methods have made significant progress, they still struggle with limited generalization to real-world scenarios due to the substantial domain gap caused by training on synthetic data. Existing methods address this issue by improving data synthesis pipelines, estimating degradation kernels, employing deep internal learning, and performing domain adaptation and regularization. Previous domain adaptation methods have sought to bridge the domain gap by learning domain-invariant knowledge in either feature or pixel space. However, these techniques often struggle to extend to low-level vision tasks within a stable and compact framework. In this paper, we show that it is possible to perform domain adaptation via the noise space using diffusion models. In particular, by leveraging the unique property of how auxiliary conditional inputs influence the multi-step denoising process, we derive a meaningful diffusion loss that guides the restoration model in progressively aligning both restored synthetic and real-world outputs with a target clean distribution. We refer to this method as denoising as adaptation. To prevent shortcuts during joint training, we present crucial strategies such as channel-shuffling layer and residual-swapping contrastive learning in the diffusion model. They implicitly blur the boundaries between conditioned synthetic and real data and prevent the reliance of the model on easily distinguishable features. Experimental results on three classical image restoration tasks, namely denoising, deblurring, and deraining, demonstrate the effectiveness of the proposed method.