MONKEY：基於鍵值激活適配器的掩碼機制，實現個性化

摘要

個性化擴散模型讓使用者能夠生成包含特定主題的新圖像，相比僅使用文字提示提供了更多控制。然而，這些模型在僅重現主題圖像而忽略文字提示時，往往表現欠佳。我們觀察到，一種流行的個性化方法——IP-Adapter，在推理過程中會自動生成遮罩，從而將主題與背景明確分割。我們提出在第二輪處理中使用這些自動生成的遮罩來遮蓋圖像標記，從而將其限制在主題而非背景上，使得文字提示能夠關注圖像的其餘部分。對於描述地點和場景的文字提示，這種方法生成的圖像既能準確描繪主題，又能完美匹配提示。我們將我們的方法與其他幾種測試時的個性化方法進行比較，發現我們的方法在提示與源圖像的對齊度上表現出色。

English

Personalizing diffusion models allows users to generate new images that incorporate a given subject, allowing more control than a text prompt. These models often suffer somewhat when they end up just recreating the subject image, and ignoring the text prompt. We observe that one popular method for personalization, the IP-Adapter automatically generates masks that we definitively segment the subject from the background during inference. We propose to use this automatically generated mask on a second pass to mask the image tokens, thus restricting them to the subject, not the background, allowing the text prompt to attend to the rest of the image. For text prompts describing locations and places, this produces images that accurately depict the subject while definitively matching the prompt. We compare our method to a few other test time personalization methods, and find our method displays high prompt and source image alignment.

MONKEY：基於鍵值激活適配器的掩碼機制，實現個性化

MONKEY: Masking ON KEY-Value Activation Adapter for Personalization

摘要

Support