递归驾驭自我改进

摘要

在模型-框架共同进化（model-harness co-evolution）的范式下，框架不仅是推理时的支撑结构，更是能够生成数据的关键组件——其执行轨迹可以塑造未来的基础模型。这催生了“框架在环学习”（harness-in-the-loop learning）的理念：对框架进行优化，既要考虑当前智能体的性能，也要兼顾其生成轨迹对未来模型训练的质量。然而，持续更新由提供方构建的框架（scaffolds）成本高昂、劳动密集。因此，我们探究是否可以通过以任务特定方式优化用户自主构建的框架，在仅需少量更新迭代且计算轻量化的前提下，提升执行轨迹的质量。为此，我们提出了递归框架自我改进（Recursive Harness Self-Improvement，RHI），该方法将框架表示为智能体循环的提示层级规范，并利用其自身修订历史中的成对反馈进行迭代优化。在涵盖量化金融、机器人学和制药领域的30个合成机器学习研究任务中，仅需少量RHI迭代，即可显著提升低推理努力（low-reasoning-effort）智能体的性能上限，甚至超越对应最高推理努力（maximum-reasoning-effort）的设置，同时将推理成本降低高达60%。我们证明，这些性能提升主要源于通过更有效的智能体间信息流来改善任务特定的上下文管理，而非依靠更长的推理轨迹。最后，我们将这一行为形式化为RHI隐含优化目标的信息论假设，表明RHI是模型-框架共同进化范式下一种实用的持续学习算法。

English

Under model--harness co-evolution, harnesses are not merely inference-time scaffolds but data-generating components whose execution traces can shape future foundation models. This motivates harness-in-the-loop learning: optimizing harnesses for both immediate agent performance and the quality of traces used for future model training. However, continually updating provider-built scaffolds is costly and labor-intensive. We therefore investigate whether optimizing user-constructed harnesses in a task-specific manner can improve execution-trace quality while remaining computationally lightweight and requiring only a few update iterations. To this end, we introduce Recursive Harness Self-Improvement (RHI), which represents the harness as a prompt-level specification of the agent loop and iteratively refines it using pairwise feedback over its own revision history. Across 30 synthetic machine-learning research tasks spanning quantitative finance, robotics, and pharmacy, a few RHI iterations suffice to substantially raise the performance ceiling of low-reasoning-effort agents, exceeding the corresponding maximum-reasoning-effort setting while reducing inference cost by up to 60%. We show that these gains arise primarily from improved task-specific context management through more effective inter-agent information flow rather than longer reasoning traces. Finally, we formalize this behavior as an information-theoretic hypothesis for RHI's implicit optimization objective, suggesting RHI as a practical algorithm for continual learning within the paradigm of model--harness co-evolution.