ReNoise：通过迭代加噪实现真实图像反演

ReNoise: Real Image Inversion Through Iterative Noising

March 21, 2024

作者: Daniel Garibi, Or Patashnik, Andrey Voynov, Hadar Averbuch-Elor, Daniel Cohen-Or

cs.AI

摘要

最近在文本引导扩散模型方面取得的进展已经解锁了强大的图像处理能力。然而，将这些方法应用于真实图像需要将图像反转到预训练扩散模型的领域。实现忠实的反转仍然是一个挑战，特别是对于最近训练用于生成具有少量去噪步骤图像的模型而言。在这项工作中，我们介绍了一种具有高质量-操作比的反转方法，提高了重建准确性而不增加操作次数。基于扩散采样过程的逆转，我们的方法在每个反转采样步骤中采用迭代去噪机制。这种机制通过迭代应用预训练扩散模型并对这些预测进行平均，改进了沿着正向扩散轨迹的预测点的逼近。我们使用各种采样算法和模型，包括最近的加速扩散模型，评估了我们的ReNoise技术的性能。通过全面的评估和比较，我们展示了它在准确性和速度方面的有效性。此外，我们通过在真实图像上展示文本驱动的图像编辑来确认我们的方法保留了可编辑性。

English

Recent advancements in text-guided diffusion models have unlocked powerful image manipulation capabilities. However, applying these methods to real images necessitates the inversion of the images into the domain of the pretrained diffusion model. Achieving faithful inversion remains a challenge, particularly for more recent models trained to generate images with a small number of denoising steps. In this work, we introduce an inversion method with a high quality-to-operation ratio, enhancing reconstruction accuracy without increasing the number of operations. Building on reversing the diffusion sampling process, our method employs an iterative renoising mechanism at each inversion sampling step. This mechanism refines the approximation of a predicted point along the forward diffusion trajectory, by iteratively applying the pretrained diffusion model, and averaging these predictions. We evaluate the performance of our ReNoise technique using various sampling algorithms and models, including recent accelerated diffusion models. Through comprehensive evaluations and comparisons, we show its effectiveness in terms of both accuracy and speed. Furthermore, we confirm that our method preserves editability by demonstrating text-driven image editing on real images.