工具之工具：具备持续改进能力的多日自主软件开发

摘要

本文研究自主软件开发——基于大语言模型（LLM）的编码智能体在无需人工干预的情况下，将高层次需求转化为完整、可运行、可用的软件系统。我们提出了 Harness-of-Harness（HoH）框架，使编码智能体能够在自主开发过程中持续改进软件。HoH 运行于现有编码智能体 harness 之上，并将其执行过程组织为迭代的“规划—编码—测试”循环。为了在各循环中维持持续改进，HoH 在修复与能力增长之间取得平衡，将开发范围界定为小而可验证的增量，将实现阶段的测试与独立评估相分离，并以可验证输出为约束而非预先规定智能体工作流。HoH 渐进式地开放交付物、角色专用工具与技能，鼓励复用而非重新创建，并维护带版本控制的项目历史。在 GameCraft-Bench、FrontierSWE 和 ProgramBench 上，针对三对 harness-模型组合（Codex with GPT-5.5、OpenCode with DeepSeek-V4-Pro、Pi with MiniMax-M3），HoH 均持续优于对应的独立 harness，经过三轮迭代后平均相对提升达 52.25%，最大提升达 82.86%。在一次持续多日、超过 70 次迭代的部署中，HoH 自主开发出一款第一人称射击游戏，该游戏具有连贯的故事情节、完整实现的核心玩法机制、可供人类玩家完整体验的可玩性、精致的视觉效果以及集成的音频。GitHub：https://github.com/Flesymeb/HarnessOfHarness 项目主页：https://flesymeb.github.io/HarnessOfHarness/

English

This paper studies autonomous software development, in which LLM-based coding agents transform high-level requirements into complete, functional, and usable software systems without human intervention. We introduce Harness-of-Harness (HoH), a framework that enables coding agents to continually improve software during autonomous development. HoH operates on existing coding-agent harnesses, and organizes their executions into iterative planning-coding-testing loops. To sustain improvement across loops, HoH balances repair with capability growth, scopes development into small and verifiable increments, separates implementation-time testing from independent evaluation, and constrains verifiable outputs rather than prescribing agent workflows. It progressively exposes deliverables, role-specific tools, and skills, encourages reuse rather than recreation, and maintains versioned project histories. On GameCraft-Bench, FrontierSWE, and ProgramBench, three harness-model pairs (Codex with GPT-5.5, OpenCode with DeepSeek-V4-Pro, and Pi with MiniMax-M3), HoH consistently outperforms the corresponding standalone harnesses, achieving an average relative gain of 52.25 percent and a maximum gain of 82.86 percent after three iterations. In a multi-day deployment with more than 70 iterations, HoH autonomously develops a first-person-shooter game, featuring a coherent storyline, fully implemented core mechanics, human-playable experience, polished visuals and integrated audio. Github: https://github.com/Flesymeb/HarnessOfHarness Project Page: https://flesymeb.github.io/HarnessOfHarness/