可驗證搜尋並非可學習的思維鏈

摘要

假設任何可透過簡短程式解決的任務，都能透過鏈式思維教給模型——寫出步驟、微調，模型就會照做——這種假設看似合理。然而，本文指出，對於一類可明確識別的程序，這個假設並不成立。測試平台包含九項推理任務，每項任務都來自確定性生成器；公開與隱藏資料集共用生成器，因此保留的資料可用來代理測試準確率。我將這些生成器逆向工程為 Python 求解器，將其呈現為鏈式思維，並蒸餾到一個基於 Nemotron（300 億參數，其中 35 億活躍參數）的排名 ≤ 32 的 LoRA 模型中。前向可計算的任務很容易安裝：查表/算術任務以及一項 8 位元布林任務的轉移成功率分別達到 ≥ 0.99 與 0.68。但數字謎題（Cryptarithm）卻不行：將其回溯搜索蒸餾出來，無論採用十一種鏈式思維設計、基於可驗證獎勵的強化學習，還是自我訓練，其表現都維持在 0.01–0.07 之間，儘管搜索求解器能回答 71% 的實例。這並非能力差距。模型在 97–100% 的行上都能正確執行算術，並在 71% 的實例中將正確的字母編碼排入前八名；但它無法將搜索視為從左到右的推導持續進行下去。微調學會了可驗證消除步驟的形式，但其判決卻變成了無條件模板，正確率僅有 16–57%（「判決即令牌」）這個天花板在 3B 到 671B 的骨幹模型、微調與提示中都維持不變；一次受控干預找到了原因：揭示密碼金鑰（此舉將推導轉為前向），使相同實例的表現從 0.03 躍升至 0.57。當某個程序的唯一解法是對無資訊結構進行搜索時，就不存在可以模仿的可信前向鏈式思維。要讓任務變得可學習，只能移除搜索，將其組合核心預先計算為目錄，並將軌跡簡化為回憶加驗證；第一名解決方案正是以此方式達到 0.92 的 Private LB 分數。被蒸餾出來的是記憶與驗證，而非搜索。

English

It is tempting to assume any task solvable by a short program can be taught to a model as its chain-of-thought: write the steps out, fine-tune, and the model follows. This paper shows the assumption fails for an identifiable class of procedures. The testbed is nine reasoning tasks, each from a deterministic generator; public and hidden splits share generators, so held-out data proxies test accuracy. I reverse-engineer the generators into Python solvers, render them as chain-of-thought, and distill into a rank-<= 32 LoRA over a 30B (3.5B-active) Nemotron model. Forward-computable tasks install readily: lookup/arithmetic and an 8-bit boolean task transfer (>= 0.99 and 0.68). Cryptarithm does not: distilling its backtracking search holds at 0.01-0.07 across eleven chain-of-thought designs, RL from verifiable rewards, and self-training, even though a search solver answers 71% of instances. This is not a capability gap. The model does the arithmetic on 97-100% of lines and ranks the correct cipher in its top eight on 71%; it cannot carry the search forward as a left-to-right derivation. Fine-tuning learns the shape of a verifiable elimination step while its verdicts become unconditional templates, correct only 16-57% of the time ("verdict-as-token"). The ceiling holds across backbones from 3B to 671B and across fine-tuning and prompting; a controlled intervention isolates the cause: revealing the cipher key, which turns the derivation forward, lifts the same instances from 0.03 to 0.57. When a procedure's only solution is search over information-free structure, no faithful forward chain-of-thought exists to imitate. The task becomes learnable only by removing the search, precomputing its combinatorial core into a catalog and reducing the trace to recall plus verification; the 1st-place solution reaches Private LB 0.92 this way. What distills is memorization and verification, not search.