Qwen-Image-2.0-RL 技術報告

摘要

我們提出Qwen-Image-2.0-RL，這是一個後訓練流程，透過基於人類回饋的強化學習（RLHF）與同策略蒸餾（OPD），改善Qwen-Image-2.0擴散模型的視覺品質與指令遵循能力。為提供可靠的獎勵訊號，我們透過微調視覺語言模型，並結合逐點評分範式與思維鏈推理，建構任務特定的複合獎勵模型。對於文字生成影像任務，獎勵模型涵蓋對齊性、美學品質與人像保真度等維度；針對影像編輯任務，獎勵系統則著重於指令遵循準確性與人臉身分保留。在此獎勵系統基礎上，我們開發一套可擴展的基於GRPO的強化學習訓練框架，結合混合無分類器引導（CFG）策略以保留預訓練知識、透過組內獎勵範圍過濾進行提示篩選，以及按類別獎勵權重校正。為融合文字生成影像與編輯兩項任務專用的強化學習策略，我們提出以同策略蒸餾作為最終訓練階段，透過軌跡層級的速度匹配將多個教師模型整合為單一學生模型。廣泛評估結果顯示，Qwen-Image-2.0-RL在Qwen-Image-Bench上取得57.84的總分（較基礎模型提升2.61分），在文字生成影像競技場中獲得1193的Elo評分（+78分），在影像編輯競技場中獲得1349分（+93分），展現出在美學品質、提示遵循與編輯準確性上的一致提升。

English

We present Qwen-Image-2.0-RL, a post-training pipeline that applies reinforcement learning from human feedback (RLHF) and on-policy distillation (OPD) to improve both the visual quality and instruction-following capability of the Qwen-Image-2.0 diffusion model. To provide reliable reward signals, we construct task-specific composite reward models by fine-tuning vision-language models with a pointwise scoring paradigm and chain-of-thought reasoning. For text-to-image generation, the reward models cover alignment, aesthetics, and portrait fidelity dimensions. For image editing tasks, the reward system addresses instruction-following accuracy and face identity preservation. Building on this reward system, we develop a scalable GRPO-based RL training framework, incorporating a hybrid classifier-free guidance (CFG) strategy to preserve pre-trained knowledge, prompt curation via intra-group reward range filtering, and per-category reward weight calibration. To merge the task-specialized RL policies for T2I and editing, we propose on-policy distillation as the final training stage, which consolidates multiple teachers into a single student model through trajectory-level velocity matching. Extensive evaluation shows that Qwen-Image-2.0-RL achieves 57.84 overall score on Qwen-Image-Bench (+2.61 over the base model), Elo ratings of 1193 in text-to-image arena (+78) and 1349 in image edit arena (+93), demonstrating consistent gains in aesthetic quality, prompt adherence, and editing accuracy.