通過受限先驗進行生成式恢復

摘要

去噪擴散模型固有的生成能力使其非常適合用於影像修復任務，其目標是在生成空間中找到與輸入影像密切相似的最佳高質量影像。我們提出了一種方法，通過向待修復的輸入影像添加噪聲，然後去噪，來適應預訓練的擴散模型進行影像修復。我們的方法基於這樣一個觀察，即生成模型的空間需要受到限制。我們通過對捕捉輸入影像特徵的一組錨定影像進行微調，來施加這種限制。有了受限空間，我們可以利用用於生成的採樣策略來進行影像修復。我們對比以前的方法進行評估，在多個真實世界的修復數據集上展示了卓越的性能，能夠保留身份和影像質量。我們還展示了一個重要且實用的個性化修復應用，其中我們使用個人相冊作為錨定影像來限制生成空間。這種方法使我們能夠產生能夠準確保留高頻細節的結果，而以前的作品無法做到。項目網頁：https://gen2res.github.io。

English

The inherent generative power of denoising diffusion models makes them well-suited for image restoration tasks where the objective is to find the optimal high-quality image within the generative space that closely resembles the input image. We propose a method to adapt a pretrained diffusion model for image restoration by simply adding noise to the input image to be restored and then denoise. Our method is based on the observation that the space of a generative model needs to be constrained. We impose this constraint by finetuning the generative model with a set of anchor images that capture the characteristics of the input image. With the constrained space, we can then leverage the sampling strategy used for generation to do image restoration. We evaluate against previous methods and show superior performances on multiple real-world restoration datasets in preserving identity and image quality. We also demonstrate an important and practical application on personalized restoration, where we use a personal album as the anchor images to constrain the generative space. This approach allows us to produce results that accurately preserve high-frequency details, which previous works are unable to do. Project webpage: https://gen2res.github.io.

通過受限先驗進行生成式恢復

Restoration by Generation with Constrained Priors

摘要

Support