MIMFlow：整合遮罩圖像建模與歸一化流實現端到端圖像生成

摘要

歸一化流（NFs）是具備精確密度估計與取樣能力的強大生成模型。然而，其嚴格的可逆性常導致模型耗盡容量處理低階像素細節，阻礙高階語意結構的捕捉。雖然遮罩圖像建模（MIM）在表徵學習上表現卓越，但與生成管線的整合多僅止於模組化且缺乏系統性。本文提出MIMFlow，一個統一且端到端的框架，能共同優化潛在語意、像素重建與生成流。透過使用VAE編碼器從遮罩圖像推斷語意潛變量，MIMFlow實現了生成任務的理論性解耦：歸一化流專注於建模簡化後的低頻語意流形，而專用解碼器則處理高頻合成。此設計有效解決歸一化流內在的容量瓶頸，使模型能優先關注全局結構連貫性，而非冗餘噪聲。在ImageNet 256×256上的實證結果顯示，MIMFlow-L達到了71.3%的線性探測準確率與2.50的FID。儘管僅使用128個令牌（比標準模型少50%），其相較於相似規模的歸一化流基線模型仍取得了32.8%的性能提升。我們的程式碼已公開於 https://github.com/MCG-NJU/MIMFlow。

English

Normalizing Flows (NFs) are powerful generative models capable of exact density estimation and sampling. However, their strict invertibility often forces the model to exhaust its capacity on low-level pixel details, hindering the capture of high-level semantic structures. While Masked Image Modeling (MIM) has excelled in representation learning, its integration into generative pipelines has remained largely modular and disjointed. In this paper, we propose MIMFlow, a unified end-to-end framework that jointly optimizes latent semantics, pixel reconstruction, and generative flow. By employing a VAE encoder to infer semantic latent from masked images, MIMFlow achieves a principled decoupling of the generative task: the Normalizing Flow focuses on modeling a simplified, low-frequency semantic manifold, while a specialized decoder handles high-frequency synthesis. This design effectively resolves the inherent capacity bottleneck of NFs, allowing the model to prioritize global structural coherence over redundant noise. Empirical results on ImageNet 256times256 show that MIMFlow-L reaches 71.3\% linear probing accuracy and an FID of 2.50. Despite using only 128 tokens (50\% fewer than standard models), it yields a 32.8\% performance gain over similar-scale NF baselines. Our code is available at https://github.com/MCG-NJU/MIMFlow.