LLaDA-Image：以完全開放的訓練配方構建強大圖像生成器

摘要

我們介紹 LLaDA-Image，這是一個統一框架，將從零開始訓練的 6B Diffusion Transformer（DiT）與以 LLaDA2.0-Mini 擴散語言模型為骨幹、並保持凍結的視覺─語言理解模組結合。我們並非從一開始便高度依賴成對的圖像─文本數據，而是先透過僅使用圖像的預訓練與中期訓練建立強大的視覺生成先驗。生成流程包含 220M 個樣本，其中 98% 為真實圖像。為了實現高效且可擴展的最佳化，我們在整個 DiT 中採用無參數 RMSNorm，並搭配 Muon 優化器。由此產生的統一模型能生成高度照片級真實的圖像，同時準確遵循細粒度的編輯指令。我們進一步將 LLaDA-Image 蒸餾為 LLaDA-Image-Turbo，使其能在 2 至 4 個採樣步驟內完成快速推論。在 Qwen-Image-Bench 上，LLaDA-Image 在英文與中文賽道分別取得 53.53 與 53.38 的總分，刷新兩個賽道中開源模型的最新最佳成績。為支持具備高效能與高效率的生成模型之後續研究，我們釋出了模型權重、訓練程式碼，以及詳細的訓練配方。

English

We introduce LLaDA-Image, a unified framework that pairs a 6B Diffusion Transformer (DiT) trained from scratch with a frozen vision-language understanding module built on the LLaDA2.0-Mini diffusion language model backbone. Instead of relying heavily on paired image-text data from the beginning, we first build a strong visual generative prior through image-only pre-training and mid-training. The generation pipeline comprises 220M samples, 98 of which are real images. For efficient and scalable optimization, we use parameter-free RMSNorm throughout the DiT together with the Muon optimizer. The resulting unified model produces highly photorealistic images while accurately following fine-grained editing instructions. We further distill LLaDA-Image into LLaDA-Image-Turbo, enabling fast inference in 2-4 sampling steps. On Qwen-Image-Bench, LLaDA-Image achieves overall scores of 53.53 and 53.38 on the English and Chinese tracks, respectively, setting a new state-of-the-art among open-source models on both tracks. To support further research on capable and efficient generative models, we release our model weights, training code, and detailed recipes.

LLaDA-Image：以完全開放的訓練配方構建強大圖像生成器

LLaDA-Image: Building Strong Image Generators with Fully Open Training Recipes

摘要

Support