面向多语言机器翻译的开放大语言模型无参考后训练

摘要

我们研究使用开源大语言模型进行多语言机器翻译的无参考后训练。从监督微调的 MiLMMT-46-v0.1 模型出发，我们应用组相对策略优化（GRPO），其奖励函数以语言识别为门控，并对两个无参考质量估计模型取平均。随后，我们对监督微调（SFT）和强化学习（RL）模型的检查点进行线性插值，得到 MiLMMT-46-v1.0。在 46 种语言上，所得模型相较于其 SFT 对应模型持续提升了翻译质量，超越了近期强力的开源基线（包括 Seed-X、HY-MT2 和 TranslateGemma），并在与所评估的专有系统（如 Google Translate、Gemini 3 Pro 和 GPT-5）的比较中取得了领先的无参考分数。我们进一步研究了同策略蒸馏，发现其能够达到、但未能超越通过 RL 结合检查点插值所实现的质量前沿。我们发布模型和代码，以促进未来研究。

English

We study reference-free post-training for multilingual machine translation with open large language models. Starting from the supervised-finetuned MiLMMT-46-v0.1 models, we apply Group Relative Policy Optimization (GRPO) with a reward that averages two reference-free quality estimation models and is gated by language identification. We then linearly interpolate the supervised fine-tuning (SFT) and reinforcement learning (RL) model checkpoints to obtain MiLMMT-46-v1.0. Across 46 languages, the resulting models consistently improve translation quality over their SFT counterparts, outperform strong recent open baselines, including Seed-X, HY-MT2, and TranslateGemma, and achieve leading reference-free scores against evaluated proprietary systems such as Google Translate, Gemini 3 Pro, and GPT-5. We further investigate on-policy distillation and find that it reaches, but does not surpass, the quality frontier achieved by RL with checkpoint interpolation. We release the models and code to facilitate future research.