低資源語言中的思考：SFT建構了什麼、RL修復了什麼、準確度無法看見什麼

摘要

取三個前沿混合專家模型（阿里巴巴、OpenAI、NVIDIA；每個約36至40億個活化參數），對它們進行微調，使其能以低資源語言推理。在準確度基準上幾乎沒有任何變化，而且在此規模下，基準本身即為噪聲：僅改變隨機種子就會讓分數移動7.7分，超過我們所測量到的所有數據與訓練配方效應。這個零結果是我們的第一項發現。真正的變化發生在準確度看不見的地方。基礎模型從不以希臘語思考：1,000條推理軌跡中有0條如此，即使問題是以希臘語提出；因此模型雖然正確作答，卻是以使用者無法閱讀、稽核或修正的形式在推理。在監督式微調（SFT）之後，所有釋出的檢查點在約98%的項目上會以問題的語言推理，其中一個模型家族使用的token數少了三分之二；經評判的語法正確性在全部四個模型上都有所提升，而一般能力與各基礎模型相差不過數分：沒有遺忘任何事物，也獲得了流暢性。我們提出六個行為維度，使這類變化可以被測量；每個維度都設有門控，以排除任何與輸出長度相關的指標，並報告我們自己的儀器如何撒謊：六次失敗，每一次都被某個對照組識破。SFT無法做到的，是修復其自身的缺陷：四分之一的回答跳過了要求的格式，答案洩漏到推理通道中，而且明確的「用英文思考」指令被遵循的時間不到一半。使用可驗證獎勵的強化學習（於訓練前預先註冊）直接修復前兩者（回退率從24%降至2.5%，洩漏率從3.5%降至0.0%，兩者皆相對於一個恆定的隨機獎勵對照組），並使第三項提升了9.1個百分點；與此同時，希臘語推理習慣在僅針對準確度的梯度下絲毫不受影響。我們釋出五個檢查點。這些儀器、對照組與預先註冊方法可遷移至任何低資源語言；希臘語是讓我們得以測量它們的案例。

English

Take three frontier mixture-of-experts models (Alibaba, OpenAI, NVIDIA; 3.6-4.0B active parameters each) and fine-tune them to reason in a low-resource language. On accuracy benchmarks almost nothing happens, and the benchmark itself is noise at this scale: changing only the random seed moves the score by 7.7 points, more than every data and recipe effect we measured. That null is our first result. The real changes live where accuracy cannot see. Base models never think in Greek: 0 of 1,000 reasoning traces, even when the question is Greek, so the model answers correctly while reasoning in a form its user cannot read, audit, or correct. After supervised fine-tuning (SFT), every released checkpoint reasons in the language of the question on ~98% of items, one family at 3x fewer tokens, with judged grammaticality improving on all four models and general ability within a few points of each base: nothing was forgotten, and fluency was gained. We propose six behavioural dimensions that make such changes measurable, each gated to reject any metric that correlates with output length, and we report how our own instruments lied: six failures, each caught by a control. What SFT cannot do is fix its own defects: a quarter of answers skip the requested format, answers leak into the reasoning channel, and an explicit "think in English" is obeyed under half the time. Reinforcement learning with verifiable rewards, pre-registered before training, fixes the first two outright (fallback 24% to 2.5%, leak 3.5% to 0.0%, both against a flat random-reward control) and moves the third (+9.1pp), while the Greek reasoning habit survives an accuracy-only gradient untouched. We release five checkpoints. The instruments, the controls and the pre-registration travel to any low-resource language; Greek is the case that let us measure them.