Meissonic：為高效率高解析度文本至圖像合成重振遮罩生成式Transformer

Meissonic: Revitalizing Masked Generative Transformers for Efficient High-Resolution Text-to-Image Synthesis

October 10, 2024

作者: Jinbin Bai, Tian Ye, Wei Chow, Enxin Song, Qing-Guo Chen, Xiangtai Li, Zhen Dong, Lei Zhu, Shuicheng Yan

cs.AI

摘要

擴散模型，如穩定擴散，已在視覺生成方面取得重大進展，然而其範式與自回歸語言模型根本不同，使統一語言-視覺模型的發展變得複雜。最近的努力，如LlamaGen，嘗試使用離散VQVAE標記來進行自回歸圖像生成，但涉及的大量標記使這種方法效率低下且緩慢。在這項工作中，我們提出了Meissonic，將非自回歸遮罩圖像建模（MIM）文本到圖像提升到與SDXL等最先進的擴散模型相當的水平。通過融入一套全面的架構創新、先進的位置編碼策略和優化的取樣條件，Meissonic顯著提高了MIM的性能和效率。此外，我們利用高質量的訓練數據，整合由人類偏好分數通知的微條件，並使用特徵壓縮層進一步增強圖像的保真度和分辨率。我們的模型不僅在生成高質量、高分辨率圖像方面與SDXL等現有模型相匹敵，甚至經常超越其表現。大量實驗驗證了Meissonic的能力，展示了其作為文本到圖像合成新標準的潛力。我們釋出了一個能夠生成1024乘以1024分辨率圖像的模型檢查點。

English

Diffusion models, such as Stable Diffusion, have made significant strides in visual generation, yet their paradigm remains fundamentally different from autoregressive language models, complicating the development of unified language-vision models. Recent efforts like LlamaGen have attempted autoregressive image generation using discrete VQVAE tokens, but the large number of tokens involved renders this approach inefficient and slow. In this work, we present Meissonic, which elevates non-autoregressive masked image modeling (MIM) text-to-image to a level comparable with state-of-the-art diffusion models like SDXL. By incorporating a comprehensive suite of architectural innovations, advanced positional encoding strategies, and optimized sampling conditions, Meissonic substantially improves MIM's performance and efficiency. Additionally, we leverage high-quality training data, integrate micro-conditions informed by human preference scores, and employ feature compression layers to further enhance image fidelity and resolution. Our model not only matches but often exceeds the performance of existing models like SDXL in generating high-quality, high-resolution images. Extensive experiments validate Meissonic's capabilities, demonstrating its potential as a new standard in text-to-image synthesis. We release a model checkpoint capable of producing 1024 times 1024 resolution images.