單圖像迭代式主體驅動生成與編輯

Single Image Iterative Subject-driven Generation and Editing

March 20, 2025

作者: Yair Shpitzer, Gal Chechik, Idan Schwartz

cs.AI

摘要

在僅有少量甚至單一主題圖像的情況下，實現圖像生成與編輯的個性化尤具挑戰性。個性化的一種常見方法是概念學習，它能夠相對快速地將主題整合到現有模型中，但當主題圖像數量較少時，生成圖像的質量往往迅速下降。通過預訓練編碼器可以提升質量，然而訓練過程限制了生成圖像僅限於訓練分佈，且耗時較長。如何在不進行訓練的情況下，從單一圖像實現圖像生成與編輯的個性化，仍是一個未解的難題。本文提出SISO，一種基於與輸入主題圖像相似度優化的新穎、無需訓練的方法。具體而言，SISO迭代生成圖像並根據與給定主題圖像的相似度損失優化模型，直至達到滿意的相似度水平，從而實現對任何圖像生成器的即插即用式優化。我們在多樣化的個人主題數據集上，針對圖像編輯和圖像生成兩項任務評估了SISO，結果顯示其在圖像質量、主題忠實度及背景保留方面相較現有方法有顯著提升。

English

Personalizing image generation and editing is particularly challenging when we only have a few images of the subject, or even a single image. A common approach to personalization is concept learning, which can integrate the subject into existing models relatively quickly, but produces images whose quality tends to deteriorate quickly when the number of subject images is small. Quality can be improved by pre-training an encoder, but training restricts generation to the training distribution, and is time consuming. It is still an open hard challenge to personalize image generation and editing from a single image without training. Here, we present SISO, a novel, training-free approach based on optimizing a similarity score with an input subject image. More specifically, SISO iteratively generates images and optimizes the model based on loss of similarity with the given subject image until a satisfactory level of similarity is achieved, allowing plug-and-play optimization to any image generator. We evaluated SISO in two tasks, image editing and image generation, using a diverse data set of personal subjects, and demonstrate significant improvements over existing methods in image quality, subject fidelity, and background preservation.