WHALE：一种简单的联合线束-权重优化方法

摘要

智能体性能由模型参数与负责管理上下文和控制流的可执行框架代码共同决定。单独优化其中任一组件，都可能让系统受制于另一保持冻结的组件，形成瓶颈：权重更新会改变哪种框架行之有效，框架更新则会改变模型展现出哪些能力。现有的联合自适应方法优化权重和文本提示，却让更广泛的框架保持固定。为此，我们提出权重-框架交替学习（Weight-Harness Alternating LEarning, WHALE），一种简单方法，交替执行两个阶段：先固定当前框架并更新模型，再在更新后的模型上搜索更优框架。这两个阶段分别由在线拒绝采样微调和 Meta-Harness 实例化。何时切换是关键的设计选择：为了将真实改进与噪声区分开，同时避免在另一组件不断变化时过度优化，WHALE 采用固定的阶段时长或基于训练信号的自适应耐心规则。在 SearchQA（搜索问答）、数学推理和国际象棋谜题三个领域中使用 Qwen3.5-2B/4B 智能体的实验表明，在最佳 mean@8 准确率上，WHALE 比仅权重、仅框架和 Fast-Slow Training 高出 4.15–24.38 个百分点。任一组件都可能成为瓶颈：在 SearchQA 中，框架搜索只需远少于仅权重优化的展开（rollout）次数即可达到后者的峰值准确率；但在数学任务上，框架搜索只有经过一次权重更新后才能提升准确率。此外，小步交错更新在准确率和展开成本上也优于先权重后框架的分阶段优化。代码已开源：https://github.com/krafton-ai/WHALE。

English

Agent performance depends jointly on the model parameters and the executable harness code that manages context and control flow. Optimizing either component in isolation can leave the system bottlenecked by its frozen counterpart: weight updates can change which harness is effective, while harness updates can change which model capabilities are exposed. Existing joint-adaptation methods optimize weights and textual prompts but leave the broader harness fixed. We propose Weight-Harness Alternating LEarning (WHALE), a simple recipe that alternates two phases: updating the model under the current harness, then searching for a better harness under the updated model. We instantiate these two phases with online rejection-sampling fine-tuning and Meta-Harness, respectively. When to switch is a key design choice: to separate real improvements from noise without over-optimizing against a changing counterpart, WHALE uses either fixed phase durations or an adaptive patience rule over training signals. Using Qwen3.5-2B/4B agents across three domains (search question answering, mathematical reasoning, and chess puzzles), WHALE outperforms weight-only, harness-only, and Fast-Slow Training by 4.15-24.38 percentage points in best mean@8 accuracy. Either component can be the bottleneck: harness search matches peak weight-only accuracy with far fewer rollouts in SearchQA, but improves math accuracy only after a weight update. Small interleaved updates also outperform stagewise weight-then-harness optimization in accuracy and rollout cost. The code is available at https://github.com/krafton-ai/WHALE.