WHALE：一種用於聯合優化束具與權重的簡易方法

摘要

智能體（agent）的效能同時取決於模型參數，以及負責管理上下文與控制流程的執行框架（harness）程式碼。若孤立地最佳化其中任一組件，系統可能因對方保持凍結不動而出現瓶頸：權重更新可能改變何種執行框架最為有效，執行框架更新則可能改變模型所被運用到的能力組合。現有的聯合適應方法會最佳化權重與文本提示（textual prompts），但卻讓更廣泛的執行框架維持不變。我們提出「權重-執行框架交替學習」（Weight-Harness Alternating LEarning，簡稱 WHALE），這是一個簡潔的訓練配方：交替進行兩個階段——先在當前執行框架下更新模型，再於更新後的模型上搜尋更佳的執行框架。我們分別以在線拒絕採樣微調（online rejection-sampling fine-tuning）與 Meta-Harness 實作這兩個階段。何時切換是一項關鍵的設計選擇：為了在不針對持續變動的另一端過度最佳化的前提下，分辨真正的改進與雜訊，WHALE 採用固定階段長度，或依據訓練訊號設定的自適應耐心規則。使用 Qwen3.5-2B/4B 智能體在三個領域（搜尋式問答、數學推理與西洋棋謎題）的實驗中，WHALE 在最佳 Mean@8 準確率上比僅權重最佳化、僅執行框架最佳化與 Fast-Slow Training 高出 4.15 至 24.38 個百分點。實驗亦顯示任一組件皆可能成為瓶頸：在 SearchQA 中，執行框架搜尋僅需遠少於權重最佳化的 rollout 次數，即可達到與僅權重最佳化相同的峰值準確率；但在數學領域，必須先經過權重更新，執行框架搜尋才能提升準確率。此外，小型交錯式更新無論在準確率或 rollout 成本上，皆優於先權重後執行框架的分階段式最佳化。程式碼已公開於 https://github.com/krafton-ai/WHALE。

English

Agent performance depends jointly on the model parameters and the executable harness code that manages context and control flow. Optimizing either component in isolation can leave the system bottlenecked by its frozen counterpart: weight updates can change which harness is effective, while harness updates can change which model capabilities are exposed. Existing joint-adaptation methods optimize weights and textual prompts but leave the broader harness fixed. We propose Weight-Harness Alternating LEarning (WHALE), a simple recipe that alternates two phases: updating the model under the current harness, then searching for a better harness under the updated model. We instantiate these two phases with online rejection-sampling fine-tuning and Meta-Harness, respectively. When to switch is a key design choice: to separate real improvements from noise without over-optimizing against a changing counterpart, WHALE uses either fixed phase durations or an adaptive patience rule over training signals. Using Qwen3.5-2B/4B agents across three domains (search question answering, mathematical reasoning, and chess puzzles), WHALE outperforms weight-only, harness-only, and Fast-Slow Training by 4.15-24.38 percentage points in best mean@8 accuracy. Either component can be the bottleneck: harness search matches peak weight-only accuracy with far fewer rollouts in SearchQA, but improves math accuracy only after a weight update. Small interleaved updates also outperform stagewise weight-then-harness optimization in accuracy and rollout cost. The code is available at https://github.com/krafton-ai/WHALE.