PhotoVerse：使用文本到圖像擴散模型進行無調整的圖像定製

摘要

個性化的文本到圖像生成已經成為一種強大且備受追捧的工具，使用戶能夠基於其特定概念和提示創建定制圖像。然而，現有的個性化方法面臨著多個挑戰，包括調整時間長、存儲需求大、每個身份需要多張輸入圖像、以及在保留身份和可編輯性方面存在限制。為了應對這些障礙，我們提出了PhotoVerse，這是一種創新方法，它在文本和圖像領域都融入了雙分支條件機制，有效控制圖像生成過程。此外，我們引入了面部身份損失作為一個新穎的組成部分，以增強在訓練過程中對身份的保留。值得注意的是，我們提出的PhotoVerse 消除了測試時間調整的需求，僅依賴於目標身份的一張面部照片，顯著降低了與圖像生成相關的資源成本。在單一訓練階段之後，我們的方法能夠在短短幾秒內生成高質量的圖像。此外，我們的方法可以生成包含各種場景和風格的多樣化圖像。廣泛的評估顯示了我們方法卓越的性能，實現了保留身份和促進可編輯性這兩個目標。項目頁面：https://photoverse2d.github.io/

English

Personalized text-to-image generation has emerged as a powerful and sought-after tool, empowering users to create customized images based on their specific concepts and prompts. However, existing approaches to personalization encounter multiple challenges, including long tuning times, large storage requirements, the necessity for multiple input images per identity, and limitations in preserving identity and editability. To address these obstacles, we present PhotoVerse, an innovative methodology that incorporates a dual-branch conditioning mechanism in both text and image domains, providing effective control over the image generation process. Furthermore, we introduce facial identity loss as a novel component to enhance the preservation of identity during training. Remarkably, our proposed PhotoVerse eliminates the need for test time tuning and relies solely on a single facial photo of the target identity, significantly reducing the resource cost associated with image generation. After a single training phase, our approach enables generating high-quality images within only a few seconds. Moreover, our method can produce diverse images that encompass various scenes and styles. The extensive evaluation demonstrates the superior performance of our approach, which achieves the dual objectives of preserving identity and facilitating editability. Project page: https://photoverse2d.github.io/

PhotoVerse：使用文本到圖像擴散模型進行無調整的圖像定製

PhotoVerse: Tuning-Free Image Customization with Text-to-Image Diffusion Models

摘要

Support