单图像迭代式主体驱动生成与编辑

Single Image Iterative Subject-driven Generation and Editing

March 20, 2025

作者: Yair Shpitzer, Gal Chechik, Idan Schwartz

cs.AI

摘要

在仅拥有少量主体图像甚至单张图像的情况下，个性化图像生成与编辑尤为困难。一种常见的个性化方法是概念学习，它能够相对快速地将主体整合到现有模型中，但当主体图像数量较少时，生成图像的质量往往会迅速下降。通过预训练编码器可以提升质量，但训练过程限制了生成范围于训练分布之内，且耗时较长。如何在不进行训练的情况下，仅凭单张图像实现个性化图像生成与编辑，仍是一个未解的难题。本文提出SISO，一种基于与输入主体图像相似度优化的新型免训练方法。具体而言，SISO迭代生成图像，并根据与给定主体图像的相似度损失优化模型，直至达到满意的相似度水平，从而实现对任何图像生成器的即插即用优化。我们在图像编辑和图像生成两项任务中，利用多样化的个人主体数据集对SISO进行了评估，结果显示其在图像质量、主体保真度及背景保留方面均显著优于现有方法。

English

Personalizing image generation and editing is particularly challenging when we only have a few images of the subject, or even a single image. A common approach to personalization is concept learning, which can integrate the subject into existing models relatively quickly, but produces images whose quality tends to deteriorate quickly when the number of subject images is small. Quality can be improved by pre-training an encoder, but training restricts generation to the training distribution, and is time consuming. It is still an open hard challenge to personalize image generation and editing from a single image without training. Here, we present SISO, a novel, training-free approach based on optimizing a similarity score with an input subject image. More specifically, SISO iteratively generates images and optimizes the model based on loss of similarity with the given subject image until a satisfactory level of similarity is achieved, allowing plug-and-play optimization to any image generator. We evaluated SISO in two tasks, image editing and image generation, using a diverse data set of personal subjects, and demonstrate significant improvements over existing methods in image quality, subject fidelity, and background preservation.