RealFill：基于参考的真实图像补全生成

RealFill: Reference-Driven Generation for Authentic Image Completion

September 28, 2023

作者: Luming Tang, Nataniel Ruiz, Qinghao Chu, Yuanzhen Li, Aleksander Holynski, David E. Jacobs, Bharath Hariharan, Yael Pritch, Neal Wadhwa, Kfir Aberman, Michael Rubinstein

cs.AI

摘要

最近在生成图像方面取得的进展带来了能够在未知区域生成高质量、可信图像内容的外部绘制和修复模型，但这些模型产生的内容是虚构的，因为它们缺乏关于真实场景的足够上下文。在这项工作中，我们提出了RealFill，一种新颖的图像补全生成方法，用于填补图像中缺失的区域，并填充应该存在的内容。RealFill是一种生成修复模型，仅使用少量场景的参考图像进行个性化。这些参考图像不必与目标图像对齐，并且可以采用截然不同的视角、光照条件、相机光圈或图像风格拍摄。一旦个性化，RealFill能够以视觉上引人注目的内容完成目标图像，这些内容忠实于原始场景。我们在一个涵盖一系列多样化和具有挑战性场景的新图像补全基准上评估了RealFill，并发现它在很大程度上优于现有方法。更多结果请查看我们的项目页面：https://realfill.github.io

English

Recent advances in generative imagery have brought forth outpainting and inpainting models that can produce high-quality, plausible image content in unknown regions, but the content these models hallucinate is necessarily inauthentic, since the models lack sufficient context about the true scene. In this work, we propose RealFill, a novel generative approach for image completion that fills in missing regions of an image with the content that should have been there. RealFill is a generative inpainting model that is personalized using only a few reference images of a scene. These reference images do not have to be aligned with the target image, and can be taken with drastically varying viewpoints, lighting conditions, camera apertures, or image styles. Once personalized, RealFill is able to complete a target image with visually compelling contents that are faithful to the original scene. We evaluate RealFill on a new image completion benchmark that covers a set of diverse and challenging scenarios, and find that it outperforms existing approaches by a large margin. See more results on our project page: https://realfill.github.io