判斷何時不宜重用：自主大型語言模型後訓練中的條件式經驗遷移

Knowing When Not to Reuse: Conditional Experience Transfer in Autonomous LLM Post-Training

August 27, 2026

作者: Tingyun Li, Wenfeng Feng, Weiqing Li, Abudukelimu Wuerkaixi, Guohua Liu, Yuewei Zhang

cs.AI

摘要

大型語言模型具備廣泛能力，但若要使其適應持續演進的領域、工具與需求，往往需要重複進行後續訓練。自主系統透過提出更新、訓練候選模型，並利用評估回饋來選擇後續提案，從而將此流程的一部分自動化。隨著證據累積，一個核心問題浮現：在後續訓練改變了父模型之後，哪些過往更新證據仍然具有可執行性？更新的效果取決於其父模型、資料與訓練階段。將過往成功視為與情境無關的許可，可能浪費運算資源；若由此產生的子模型被晉升，也可能使得後續訓練軌跡劣化。我們將此問題形式化為條件式經驗遷移，並提出邊界校準干預遷移法（Boundary-Calibrated Intervention Transfer, BCIT）。該方法在改變權重的訓練之前授權經驗重用：將觀測到的效果與其來源情境綁定，檢查適用條件，否決具有明確硬衝突的候選項，並在必要時透過有界的訓練試行取得當前狀態證據。完整訓練後的候選項仍須遵循共通的採用規則，且只有觀測到的事件會擴充記憶。在一個經財務推理、文字轉SQL與函式呼叫等任務調適的4B模型上，候選更新在所評估的情境中展現出異質的目標效果與保留效果。在候選項、證據與運算量匹配的條件下，BCIT相較所評估的替代方法，授權了較少的有害更新，並在相同預算下達成更高的最終模型品質。這些結果支持將經驗授權視為自主後續訓練中的一個獨立問題。

English

Large language models offer broad capabilities, but adapting them to evolving domains, tools, and requirements often entails repeated post-training. Autonomous systems automate parts of this process by proposing updates, training candidates, and using evaluation feedback to select subsequent proposals. As evidence accumulates, a central problem emerges: which past update evidence remains actionable after subsequent training has changed the parent model? An update's effect depends on its parent, data, and training stage. Treating past success as context-free permission can waste compute. If the resulting child is promoted, it can also degrade the subsequent training trajectory. We formulate this problem as conditional experience transfer and introduce Boundary-Calibrated Intervention Transfer (BCIT), a method that authorizes experience reuse before weight-changing training. BCIT binds an observed effect to its source context, checks applicability conditions, vetoes candidates with named hard conflicts, and obtains current-state evidence through a bounded training trial when needed. Fully trained candidates still face a shared adoption rule, and only observed events extend memory. On one 4B model adapted across finance reasoning, text-to-SQL, and function calling, candidate updates exhibit heterogeneous target and retention effects across the evaluated contexts. Under matched candidates, evidence, and compute, BCIT authorizes fewer harmful updates and attains higher equal-budget final-model quality than the evaluated alternatives. These results support treating experience authorization as a distinct problem in autonomous post-training.