單提示單故事：使用單個提示進行免費午餐一致的文本到圖像生成

One-Prompt-One-Story: Free-Lunch Consistent Text-to-Image Generation Using a Single Prompt

January 23, 2025

作者: Tao Liu, Kai Wang, Senmao Li, Joost van de Weijer, Fahad Shahbaz Khan, Shiqi Yang, Yaxing Wang, Jian Yang, Ming-Ming Cheng

cs.AI

摘要

文本到圖像生成模型可以從輸入提示創建高質量圖像。然而，它們在支持保持故事中保持身份的一致生成方面存在困難。解決這個問題的現有方法通常需要在大型數據集上進行廣泛訓練，或對原始模型架構進行額外修改。這限制了它們在不同領域和多樣擴散模型配置中的應用。在本文中，我們首先觀察到語言模型的固有能力，即所謂的上下文一致性，通過單個提示理解身份。受固有上下文一致性的啟發，我們提出了一種新的無需訓練的方法，用於一致的文本到圖像（T2I）生成，稱為“一提示一故事”（1Prompt1Story）。我們的方法1Prompt1Story將所有提示串聯為T2I擴散模型的單個輸入，最初保留角色身份。然後，我們使用兩種新技術進行生成過程的精煉：奇異值重新加權和保持身份的交叉注意力，確保每個幀更好地與輸入描述對齊。在我們的實驗中，我們通過定量指標和定性評估將我們的方法與各種現有的一致T2I生成方法進行比較，以展示其有效性。代碼可在https://github.com/byliutao/1Prompt1Story找到。

English

Text-to-image generation models can create high-quality images from input prompts. However, they struggle to support the consistent generation of identity-preserving requirements for storytelling. Existing approaches to this problem typically require extensive training in large datasets or additional modifications to the original model architectures. This limits their applicability across different domains and diverse diffusion model configurations. In this paper, we first observe the inherent capability of language models, coined context consistency, to comprehend identity through context with a single prompt. Drawing inspiration from the inherent context consistency, we propose a novel training-free method for consistent text-to-image (T2I) generation, termed "One-Prompt-One-Story" (1Prompt1Story). Our approach 1Prompt1Story concatenates all prompts into a single input for T2I diffusion models, initially preserving character identities. We then refine the generation process using two novel techniques: Singular-Value Reweighting and Identity-Preserving Cross-Attention, ensuring better alignment with the input description for each frame. In our experiments, we compare our method against various existing consistent T2I generation approaches to demonstrate its effectiveness through quantitative metrics and qualitative assessments. Code is available at https://github.com/byliutao/1Prompt1Story.