Qwen-Image-2.0-RL 技术报告

摘要

我们提出了Qwen-Image-2.0-RL，这是一种后训练流程，通过应用基于人类反馈的强化学习（RLHF）和策略蒸馏（OPD）来提升Qwen-Image-2.0扩散模型的视觉质量和指令跟随能力。为了提供可靠的奖励信号，我们通过微调视觉语言模型，采用逐点评分范式和思维链推理，构建了特定任务的复合奖励模型。对于文本到图像生成任务，奖励模型覆盖了对齐度、美学和人像保真度等多个维度；对于图像编辑任务，奖励系统则关注指令跟随准确性和人脸身份保持。基于此奖励系统，我们开发了一个可扩展的基于GRPO的RL训练框架，该框架融合了混合无分类器引导（CFG）策略以保留预训练知识，通过组内奖励范围过滤进行提示筛选，以及按类别进行奖励权重校准。为了合并T2I和编辑任务专用的RL策略，我们将策略蒸馏作为最终训练阶段，通过轨迹级速度匹配将多个教师模型整合到单一学生模型中。大量评估表明，Qwen-Image-2.0-RL在Qwen-Image-Bench上取得了57.84的总分（较基础模型提升2.61分），在文本到图像竞技场中获得1193的Elo评分（提升78分），在图像编辑竞技场中获得1349的Elo评分（提升93分），在美学质量、提示遵循度和编辑准确性方面均展现出持续提升。

English

We present Qwen-Image-2.0-RL, a post-training pipeline that applies reinforcement learning from human feedback (RLHF) and on-policy distillation (OPD) to improve both the visual quality and instruction-following capability of the Qwen-Image-2.0 diffusion model. To provide reliable reward signals, we construct task-specific composite reward models by fine-tuning vision-language models with a pointwise scoring paradigm and chain-of-thought reasoning. For text-to-image generation, the reward models cover alignment, aesthetics, and portrait fidelity dimensions. For image editing tasks, the reward system addresses instruction-following accuracy and face identity preservation. Building on this reward system, we develop a scalable GRPO-based RL training framework, incorporating a hybrid classifier-free guidance (CFG) strategy to preserve pre-trained knowledge, prompt curation via intra-group reward range filtering, and per-category reward weight calibration. To merge the task-specialized RL policies for T2I and editing, we propose on-policy distillation as the final training stage, which consolidates multiple teachers into a single student model through trajectory-level velocity matching. Extensive evaluation shows that Qwen-Image-2.0-RL achieves 57.84 overall score on Qwen-Image-Bench (+2.61 over the base model), Elo ratings of 1193 in text-to-image arena (+78) and 1349 in image edit arena (+93), demonstrating consistent gains in aesthetic quality, prompt adherence, and editing accuracy.