Meissonic：为高效的高分辨率文本到图像合成注入新活力的遮蔽式生成变换器

Meissonic: Revitalizing Masked Generative Transformers for Efficient High-Resolution Text-to-Image Synthesis

October 10, 2024

作者: Jinbin Bai, Tian Ye, Wei Chow, Enxin Song, Qing-Guo Chen, Xiangtai Li, Zhen Dong, Lei Zhu, Shuicheng Yan

cs.AI

摘要

扩散模型，例如稳定扩散，在视觉生成方面取得了重大进展，然而其范式与自回归语言模型根本不同，使得统一语言-视觉模型的发展变得复杂。最近的工作，如LlamaGen，尝试使用离散VQVAE标记进行自回归图像生成，但涉及的大量标记使得这种方法效率低下且速度缓慢。在这项工作中，我们提出了Meissonic，将非自回归遮蔽图像建模（MIM）文本到图像提升到与诸如SDXL等最先进扩散模型相媲美的水平。通过整合全面的架构创新、先进的位置编码策略和优化的采样条件，Meissonic显著提高了MIM的性能和效率。此外，我们利用高质量的训练数据，整合由人类偏好分数提供的微条件，并采用特征压缩层进一步增强图像的保真度和分辨率。我们的模型不仅在生成高质量、高分辨率图像方面与现有模型如SDXL相匹敌，甚至常常超越。大量实验证实了Meissonic的能力，展示了其作为文本到图像合成新标准的潜力。我们发布了一个能够生成1024乘以1024分辨率图像的模型检查点。

English

Diffusion models, such as Stable Diffusion, have made significant strides in visual generation, yet their paradigm remains fundamentally different from autoregressive language models, complicating the development of unified language-vision models. Recent efforts like LlamaGen have attempted autoregressive image generation using discrete VQVAE tokens, but the large number of tokens involved renders this approach inefficient and slow. In this work, we present Meissonic, which elevates non-autoregressive masked image modeling (MIM) text-to-image to a level comparable with state-of-the-art diffusion models like SDXL. By incorporating a comprehensive suite of architectural innovations, advanced positional encoding strategies, and optimized sampling conditions, Meissonic substantially improves MIM's performance and efficiency. Additionally, we leverage high-quality training data, integrate micro-conditions informed by human preference scores, and employ feature compression layers to further enhance image fidelity and resolution. Our model not only matches but often exceeds the performance of existing models like SDXL in generating high-quality, high-resolution images. Extensive experiments validate Meissonic's capabilities, demonstrating its potential as a new standard in text-to-image synthesis. We release a model checkpoint capable of producing 1024 times 1024 resolution images.