測試時的AI4AI：透過調控機制從強到弱的能力遷移

摘要

近期關於蒸餾的研究，常透過教師強制、在策略蒸餾及相關的訓練時間方法，更新較小模型的參數，將大型模型的能力轉移給較小模型。在本論文中，我們探討此類轉移是否可以在測試時間發生。我們研究強對弱鷹架：較強的建構者模型是否可以建構推論時間框架，在沒有任何參數更新的情況下，幫助較弱的目標模型更可靠地解決任務。我們使用四個具代表性的心智理論基準，每個建構者模型使用 5% 的資料作為驗證集，透過多輪迭代來精煉其框架，之後在完整測試集上評估最終定稿的框架。實證上，此種測試時間能力轉移非常有效，將目標模型的平均效能從 0.49 提升至 0.91，幾乎翻倍。我們的分析顯示，這些增益主要來自於將不穩定的模型推理卸載到確定性程式碼、基準特定的路由，以及嚴格的答案格式強制執行，而非來自於鼓勵目標模型進行更廣泛的推理或更廣泛的抽樣。我們進一步發現，建構者模型的推理投入會單調地提升框架品質；相對於建構者模型自身的能力，平台效應是有限的；而較弱的目標模型獲得的增益最大。這些結果顯示，推論時間框架設計是傳統訓練時間蒸餾的重要補充，使得強模型無需重新訓練即可將認知結構轉移給較弱的模型。

English

Recent work on distillation transfers the capabilities of large models to smaller ones often by updating the latter's parameters, through teacher forcing, on-policy distillation, and related training-time methods. In this paper, we ask whether such transfer can instead occur at test time. We study strong-to-weak scaffolding: whether a stronger builder model can construct inference-time harnesses that help a weaker target model solve tasks more reliably without any parameter updates. Using four representative Theory-of-Mind benchmarks, each builder model uses 5% of the data as a validation set to iteratively refine its harness over multiple rounds, after which the finalized harness is evaluated on the full test set. Empirically, this form of test-time capability transfer is highly effective, nearly doubling average target-model performance from 0.49 to 0.91. Our analysis shows that the gains come primarily from offloading unstable model reasoning into deterministic code, benchmark-specific routing, and strict answer-format enforcement, rather than from encouraging the target model to reason more extensively or sample more broadly. We further find that builder-model reasoning effort improves harness quality monotonically, platform effects are modest relative to the builder model's own capability, and weaker target models receive the largest gains. These results suggest that inference-time harness design is an important complement to conventional training-time distillation, enabling strong models to transfer cognitive structure to weaker models without retraining.