遞迴駕馭自我改進

摘要

在模型—框架協同演化下，框架不僅是推理時的支撐，更是數據生成組件，其執行軌跡可塑造未來的基礎模型。這推動了框架在環學習：優化框架以兼顧即時代理效能與未來模型訓練所用軌跡的品質。然而，持續更新提供者構建的支撐成本高昂且耗費人力。因此我們研究，以任務特定方式優化使用者構建的框架，是否能在保持運算輕量且僅需少量更新迭代的情況下，提升執行軌跡品質。為此，我們引入遞迴框架自我改進（RHI），將框架表示為代理循環的提示層級規範，並利用其自身修訂歷史上的成對反饋進行迭代精煉。在橫跨量化金融、機器人學與藥學的30個合成機器學習研究任務中，僅需少數RHI迭代即可大幅提升低推理努力代理的效能上限，超越對應的最大推理努力設定，同時將推理成本降低多達60%。我們證明這些增益主要源自透過更有效的代理間資訊流來改善任務特定上下文管理，而非更長的推理軌跡。最後，我們將此行為形式化為RHI隱含最佳化目標的資訊理論假說，提出RHI作為模型—框架協同演化範式下持續學習的實用演算法。

English

Under model--harness co-evolution, harnesses are not merely inference-time scaffolds but data-generating components whose execution traces can shape future foundation models. This motivates harness-in-the-loop learning: optimizing harnesses for both immediate agent performance and the quality of traces used for future model training. However, continually updating provider-built scaffolds is costly and labor-intensive. We therefore investigate whether optimizing user-constructed harnesses in a task-specific manner can improve execution-trace quality while remaining computationally lightweight and requiring only a few update iterations. To this end, we introduce Recursive Harness Self-Improvement (RHI), which represents the harness as a prompt-level specification of the agent loop and iteratively refines it using pairwise feedback over its own revision history. Across 30 synthetic machine-learning research tasks spanning quantitative finance, robotics, and pharmacy, a few RHI iterations suffice to substantially raise the performance ceiling of low-reasoning-effort agents, exceeding the corresponding maximum-reasoning-effort setting while reducing inference cost by up to 60%. We show that these gains arise primarily from improved task-specific context management through more effective inter-agent information flow rather than longer reasoning traces. Finally, we formalize this behavior as an information-theoretic hypothesis for RHI's implicit optimization objective, suggesting RHI as a practical algorithm for continual learning within the paradigm of model--harness co-evolution.