Harness-of-Harness：具持續改進能力的多日自主軟體開發

摘要

本文研究自主軟體開發，在此類開發中，基於大型語言模型（LLM）的編碼代理能於無人為介入的情況下，將高階需求轉化為完整、功能正常且可用的軟體系統。我們提出了 Harness-of-Harness（HoH）框架，使編碼代理能夠在自主開發過程中持續改進軟體。HoH 以現有的編碼代理框架為基礎，將其執行過程組織為「規劃—編碼—測試」的迭代循環。為了在迭代間維持改進，HoH 平衡了修復與能力增長之間的關係，將開發範疇切分為小而可驗證的增量，將實作期間的測試與獨立評估分離，並以可驗證的輸出作為約束，而非預先規定代理的工作流程。此外，它逐步提供可交付成果、角色專用工具與技能，鼓勵重用而非重新建立，並維護具有版本管理的專案歷史。在 GameCraft-Bench、FrontierSWE 與 ProgramBench 上，三組框架—模型配對（Codex 搭配 GPT-5.5、OpenCode 搭配 DeepSeek-V4-Pro，以及 Pi 搭配 MiniMax-M3）中，HoH 的表現一貫優於對應的獨立框架，在三輪迭代後達成了 52.25% 的平均相對增益，最高增益則達 82.86%。在一場歷時多日、超過 70 次迭代的部署中，HoH 自主開發了一款第一人稱射擊遊戲，具備連貫的故事情節、完整實作的核心機制、可供真人遊玩的體驗、精緻的視覺效果與整合的音訊。GitHub：https://github.com/Flesymeb/HarnessOfHarness 專案頁面：https://flesymeb.github.io/HarnessOfHarness/

English

This paper studies autonomous software development, in which LLM-based coding agents transform high-level requirements into complete, functional, and usable software systems without human intervention. We introduce Harness-of-Harness (HoH), a framework that enables coding agents to continually improve software during autonomous development. HoH operates on existing coding-agent harnesses, and organizes their executions into iterative planning-coding-testing loops. To sustain improvement across loops, HoH balances repair with capability growth, scopes development into small and verifiable increments, separates implementation-time testing from independent evaluation, and constrains verifiable outputs rather than prescribing agent workflows. It progressively exposes deliverables, role-specific tools, and skills, encourages reuse rather than recreation, and maintains versioned project histories. On GameCraft-Bench, FrontierSWE, and ProgramBench, three harness-model pairs (Codex with GPT-5.5, OpenCode with DeepSeek-V4-Pro, and Pi with MiniMax-M3), HoH consistently outperforms the corresponding standalone harnesses, achieving an average relative gain of 52.25 percent and a maximum gain of 82.86 percent after three iterations. In a multi-day deployment with more than 70 iterations, HoH autonomously develops a first-person-shooter game, featuring a coherent storyline, fully implemented core mechanics, human-playable experience, polished visuals and integrated audio. Github: https://github.com/Flesymeb/HarnessOfHarness Project Page: https://flesymeb.github.io/HarnessOfHarness/