低资源语言中的思考：监督微调构建了什么，强化学习修复了什么，准确率无法看到什么

摘要

以前沿混合专家模型（阿里巴巴、OpenAI、NVIDIA；每个模型活跃参数为36–40亿）为例，对它们进行微调，使其以低资源语言进行推理。在准确性基准测试上几乎没有任何变化；在此规模下，基准本身即是噪声：仅改变随机种子就会使分数变动7.7个百分点，超过我们所测量的所有数据与训练方案影响之和。这一零结果就是我们的第一个发现。真正的变化发生在准确性无法察觉之处。基础模型从不用希腊语思考：在1000条推理轨迹中为0，即使问题是用希腊语提出的，因此模型虽然回答正确，但其推理形式用户无法阅读、审计或纠正。经过监督微调（SFT）后，所有发布的检查点在约98%的项目上使用与问题相同的语言进行推理；其中一个模型家族所用token数减少至原来的三分之一；四个模型的语法性评估均有提升，通用能力与各自基础模型相比仅相差几个百分点：既没有遗忘任何能力，又获得了流畅性。我们提出了六个行为维度，使此类变化可被测量；每个维度都设有门控，以排除任何与输出长度相关的指标。我们还报告了自身工具如何“撒谎”：六类失败，每类都被对照组所识别。而SFT无法修复自身的缺陷：四分之一的回答未采用所要求的格式，答案泄漏到推理通道中，且明确要求“用英语思考”时，模型服从的时间不足一半。采用可验证奖励的强化学习（在训练前预先注册）彻底修复了前两个问题（格式回退从24%降至2.5%，泄漏从3.5%降至0.0%，均对比平坦随机奖励对照组），并使第三个问题改善9.1个百分点；同时，希腊语推理习惯在仅以准确性为导向的梯度训练中完好保留。我们发布了五个检查点。这些工具、对照组和预注册方案可迁移至任何低资源语言；希腊语正是让我们能够对其进行测量的案例。

English

Take three frontier mixture-of-experts models (Alibaba, OpenAI, NVIDIA; 3.6-4.0B active parameters each) and fine-tune them to reason in a low-resource language. On accuracy benchmarks almost nothing happens, and the benchmark itself is noise at this scale: changing only the random seed moves the score by 7.7 points, more than every data and recipe effect we measured. That null is our first result. The real changes live where accuracy cannot see. Base models never think in Greek: 0 of 1,000 reasoning traces, even when the question is Greek, so the model answers correctly while reasoning in a form its user cannot read, audit, or correct. After supervised fine-tuning (SFT), every released checkpoint reasons in the language of the question on ~98% of items, one family at 3x fewer tokens, with judged grammaticality improving on all four models and general ability within a few points of each base: nothing was forgotten, and fluency was gained. We propose six behavioural dimensions that make such changes measurable, each gated to reject any metric that correlates with output length, and we report how our own instruments lied: six failures, each caught by a control. What SFT cannot do is fix its own defects: a quarter of answers skip the requested format, answers leak into the reasoning channel, and an explicit "think in English" is obeyed under half the time. Reinforcement learning with verifiable rewards, pre-registered before training, fixes the first two outright (fallback 24% to 2.5%, leak 3.5% to 0.0%, both against a flat random-reward control) and moves the third (+9.1pp), while the Greek reasoning habit survives an accuracy-only gradient untouched. We release five checkpoints. The instruments, the controls and the pre-registration travel to any low-resource language; Greek is the case that let us measure them.