Nemotron-Labs-Diffusion-Image: 推进面向高分辨率图像合成的掩码离散扩散

摘要

我们提出了Nemotron-Labs-Diffusion-Image，这是一种用于高分辨率文本到图像合成的最新掩蔽离散扩散模型（MDM）。与先前在掩蔽图像生成方面的工作相比，Nemotron-Labs-Diffusion-Image解决了两个关键挑战。首先，与在整个图像上逐步细化潜在表示的连续扩散模型不同，标准MDM缺乏自我修正能力，因为离散标记一旦被取消掩蔽就无法修改。其次，尽管增加离散图像分词器的词汇量可以提高重建保真度，但由于每个标记的训练信号变得日益稀疏，这给生成建模带来了优化困难。为应对第一个挑战，Nemotron-Labs-Diffusion-Image引入了一种标记编辑机制，使模型能够在推理过程中动态修改已取消掩蔽的标记，类似于雕刻家不断完善其作品。为解决第二个挑战，我们提出了一种分组交叉熵（GCE）目标函数，该函数为嵌入空间中邻近真实值的标记分配正向学习信号，从而缓解信号稀疏性问题。为进一步提升训练效率，我们为GCE实现了一个自定义融合算子，显著减少了大词汇量场景下的显存占用。实验结果表明，这些创新显著提升了掩蔽离散图像生成器的训练效率和图像保真度，在GenEval上达到0.90分，DPG上为86.9分，HPSv3为10.76分。

English

We propose Nemotron-Labs-Diffusion-Image, a state-of-the-art masked discrete diffusion model (MDM) for high-resolution text-to-image synthesis. Compared with prior work on masked image generation, Nemotron-Labs-Diffusion-Image addresses two key challenges. First, unlike continuous diffusion models which progressively refine latent representations across the entire image, standard MDMs lack self-correcting capability because discrete tokens cannot be modified once they are unmasked. Second, although increasing the vocabulary size of discrete image tokenizers improves reconstruction fidelity, it introduces optimization difficulties for generative modeling as the per-token training signal becomes increasingly sparse. To address the first challenge, Nemotron-Labs-Diffusion-Image incorporates a token-editing mechanism that enables the model to dynamically revise already-unmasked tokens during inference, similar to how a sculptor iteratively refines their work. To tackle the second challenge, we propose a Grouped Cross-Entropy (GCE) objective that assigns positive learning signals to tokens neighboring the ground truth in embedding space, thereby alleviating signal sparsity. To further improve training efficiency, we implement a custom fused operator for GCE that significantly reduces VRAM usage in large-vocabulary settings. Experimental results demonstrate that these innovations substantially improve both training efficiency and image fidelity of masked discrete image generators, achieving a score of 0.90 on GenEval, 86.9 on DPG and 10.76 of HPSv3.