無參考開放式大型語言模型於多語言機器翻譯之後訓練

摘要

我們研究以開放式大型語言模型進行多語言機器翻譯的無參考後訓練。從經過監督式微調的 MiLMMT-46-v0.1 模型出發，我們應用群體相對策略最佳化（GRPO），其獎勵為兩個無參考品質評估模型的平均，並以語言識別作為門控。接著，我們對監督式微調（SFT）與強化學習（RL）模型檢查點進行線性插值，得到 MiLMMT-46-v1.0。在 46 種語言上，所得模型相較於其 SFT 對應版本一致地提升了翻譯品質，勝過近期強大的開放式基準模型（包括 Seed-X、HY-MT2 與 TranslateGemma），並且在針對 Google Translate、Gemini 3 Pro 與 GPT-5 等受評專有系統的評估中取得領先的無參考評分。我們進一步探討在策略蒸餾，發現其可達到、但未能超越透過 RL 搭配檢查點插值所實現的品質前沿。我們釋出模型與程式碼，以促進未來研究。

English

We study reference-free post-training for multilingual machine translation with open large language models. Starting from the supervised-finetuned MiLMMT-46-v0.1 models, we apply Group Relative Policy Optimization (GRPO) with a reward that averages two reference-free quality estimation models and is gated by language identification. We then linearly interpolate the supervised fine-tuning (SFT) and reinforcement learning (RL) model checkpoints to obtain MiLMMT-46-v1.0. Across 46 languages, the resulting models consistently improve translation quality over their SFT counterparts, outperform strong recent open baselines, including Seed-X, HY-MT2, and TranslateGemma, and achieve leading reference-free scores against evaluated proprietary systems such as Google Translate, Gemini 3 Pro, and GPT-5. We further investigate on-policy distillation and find that it reaches, but does not surpass, the quality frontier achieved by RL with checkpoint interpolation. We release the models and code to facilitate future research.