RankE：具解碼器共同演化之端到端後訓練離散文字到圖像生成

摘要

离散自回归（AR）文本到图像（T2I）模型将VQ分词器与AR策略配对，当前的后训练管线仅优化策略而冻结VQ解码器。近期以REPA-E为代表的扩散T2I研究表明，VAE本身构成关键的对齐瓶颈，然而离散AR模型中尚无类似研究。我们发现，仅优化策略会引发潜在协变量偏移：随着策略演化，所生成的标记分布偏离了解码器训练时所用的真实分布，导致奖励分数提升而解码图像质量下降。为解决这一不匹配问题，我们提出RankE——首个用于离散T2I生成的端到端后训练框架。RankE并非在固定解码器下优化策略，而是通过交替优化让两个模块协同演进：每个模块在最大化基于排名的对齐目标的同时，受其参数空间内保持稳定性的锚点正则化。这种协同演进打破了困扰冻结解码器方法的质量-对齐权衡：在LlamaGen-XL（775M）上，标准RL提升了CLIP但恶化了FID，而RankE同时改善了二者（在MS-COCO 30K上，FID 15.21，CLIP 33.76）。在Janus-Pro（1B）上的一致增益证实，解码器协同演进能够可靠地将奖励优化转化为像素级的质量提升。

English

Discrete autoregressive (AR) text-to-image (T2I) models pair a VQ tokenizer with an AR policy, and current post-training pipelines optimize only the policy while keeping the VQ decoder frozen. Recent diffusion T2I work, exemplified by REPA-E, has shown that the VAE itself constitutes a key alignment bottleneck, yet no analogous investigation exists for discrete AR models. We show that policy-only optimization induces Latent Covariate Shift: as the policy evolves, the resulting token distribution diverges from the ground-truth distribution on which the decoder was trained, such that reward scores improve while decoded image quality degrades. To address this mismatch, we propose RankE, the first end-to-end post-training framework for discrete T2I generation. Rather than optimizing the policy against a fixed decoder, RankE co-evolves both components through alternating optimization: each module maximizes a ranking-based alignment objective while being regularized by a stability-preserving anchor suited to its parameter space. This co-evolution breaks the fidelity--alignment trade-off that plagues frozen-decoder approaches: on LlamaGen-XL (775M), standard RL improves CLIP but degrades FID, whereas RankE improves both simultaneously (FID 15.21, CLIP 33.76 on MS-COCO 30K). Consistent gains on Janus-Pro (1B) confirm that decoder co-evolution reliably converts reward optimization into pixel-space quality improvements.