AdaIR：通過頻率挖掘和調製的自適應全能圖像修復

AdaIR: Adaptive All-in-One Image Restoration via Frequency Mining and Modulation

March 21, 2024

作者: Yuning Cui, Syed Waqas Zamir, Salman Khan, Alois Knoll, Mubarak Shah, Fahad Shahbaz Khan

cs.AI

摘要

在影像獲取過程中，常常會引入各種形式的退化，包括噪音、霧霾和雨水。這些退化通常源於相機固有的限制或不利的環境條件。為了從退化版本中恢復乾淨的影像，已經開發了許多專門的修復方法，每種方法針對特定類型的退化。最近，全能算法通過在單一模型中處理不同類型的退化而不需要事先了解輸入退化類型而受到廣泛關注。然而，這些方法純粹在空間域中運作，並沒有深入探討與不同退化類型固有的不同頻率變化。為了彌補這一缺口，我們提出了一種基於頻率挖掘和調製的自適應全能影像修復網絡。我們的方法是基於一個觀察，即不同的退化類型會影響不同頻率子帶上的影像內容，因此需要針對每個修復任務進行不同的處理。具體來說，我們首先從輸入特徵中挖掘低頻和高頻信息，受到退化影像的自適應解耦譜的引導。然後，提取的特徵通過雙向運算子進行調製，以促進不同頻率成分之間的交互作用。最後，調製的特徵與原始輸入合併，進行逐步引導的修復。通過這種方法，模型實現了根據不同輸入退化強調信息頻率子帶的自適應重建。大量實驗表明，所提出的方法在不同影像修復任務上實現了最先進的性能，包括降噪、去霧、去雨、運動去模糊和低光影像增強。我們的代碼可在https://github.com/c-yn/AdaIR 上找到。

English

In the image acquisition process, various forms of degradation, including noise, haze, and rain, are frequently introduced. These degradations typically arise from the inherent limitations of cameras or unfavorable ambient conditions. To recover clean images from degraded versions, numerous specialized restoration methods have been developed, each targeting a specific type of degradation. Recently, all-in-one algorithms have garnered significant attention by addressing different types of degradations within a single model without requiring prior information of the input degradation type. However, these methods purely operate in the spatial domain and do not delve into the distinct frequency variations inherent to different degradation types. To address this gap, we propose an adaptive all-in-one image restoration network based on frequency mining and modulation. Our approach is motivated by the observation that different degradation types impact the image content on different frequency subbands, thereby requiring different treatments for each restoration task. Specifically, we first mine low- and high-frequency information from the input features, guided by the adaptively decoupled spectra of the degraded image. The extracted features are then modulated by a bidirectional operator to facilitate interactions between different frequency components. Finally, the modulated features are merged into the original input for a progressively guided restoration. With this approach, the model achieves adaptive reconstruction by accentuating the informative frequency subbands according to different input degradations. Extensive experiments demonstrate that the proposed method achieves state-of-the-art performance on different image restoration tasks, including denoising, dehazing, deraining, motion deblurring, and low-light image enhancement. Our code is available at https://github.com/c-yn/AdaIR.