CodeMidas：從程式碼本身擴展代理式編碼強化學習環境

CodeMidas: Scaling Agentic Coding RL Environments from Code Itself

September 18, 2026

作者: Bowen Ye, Lei Li, Shicheng Li, Zihao Yue, Linghao Zhang, Hanglong Lv, Yuanxin Liu, Wenhan Ma, Hao Tian, Rang Li, Jinhao Dong, Yikai Zhao, Xiangwei Deng, Hailin Zhang, Liang Zhao, Qi Liu, Lingpeng Kong, Tong Yang, Fuli Luo

cs.AI

摘要

透過強化學習（RL）訓練具備能力的程式編寫代理，需要具有可靠驗證器的多樣化任務。開源程式碼庫提供了這類任務的豐富來源，而現有方法通常依賴議題與提交等開發產物，限制了可擷取的任務範圍。為了更好地擴展 RL 環境，我們提出 CodeMidas，一套代理式管線，能將現有程式碼庫中已實作的功能轉換為可執行的 RL 環境，且僅以原始碼作為任務特定輸入。CodeMidas 將代理運算分配到環境建構的每個階段：代理探索已實作功能以制定行為規格，建構以原始程式碼執行為基礎的測試，並透過執行檢查與反覆解答 rollout 來驗證和篩選候選任務。產生的資料集包含來自 3,185 個開源程式碼庫的 5,545 個訓練任務，涵蓋 23 種程式語言與 15 個技術領域。以 GRPO 在這些任務上訓練 MiMo-V2.5，可提升五個多樣化基準測試的效能，涵蓋問題修復（DeepSWE +11.7%）、全程式建構（ProgramBench +17%）與終端機工作（Terminal-Bench v2.1 +8.5%）。消融實驗顯示，增加高品質訓練任務的數量可提升效能。軌跡分析顯示，經 RL 訓練的代理展現出更好的行為，例如增加程式碼庫探索，以及更多樣化的自我驗證。這些結果確立了原始碼可作為可擴展的基礎，用以建構能提升程式編寫代理在多元軟體任務上表現的 RL 環境。

English

Training capable coding agents via reinforcement learning (RL) requires diverse tasks with reliable verifiers. Open-source codebases offer a rich source of such tasks, while existing methods typically rely on development artifacts such as issues and commits, limiting the range of tasks that can be extracted. To better scale RL environments, we present CodeMidas, an agentic pipeline that turns implemented functionality in existing codebases into executable RL environments using source code as its only task-specific input. CodeMidas allocates agentic compute to every stage of environment construction: agents explore implemented functionality to formulate behavioral specifications, construct tests grounded in execution of the original code, and validate and filter candidate tasks through execution checks and repeated solution rollouts. The resulting dataset has 5,545 training tasks from 3,185 open-source codebases spanning 23 programming languages and 15 technical domains. Training MiMo-V2.5 on these tasks with GRPO improves performance on all five diverse benchmarks, covering issue repair (DeepSWE + 11.7%), whole-program construction (ProgramBench +17%), and terminal work (Terminal-Bench v2.1 +8.5%). Ablations show that increasing the number of high-quality training tasks improves performance. Trajectory analysis shows the RL-trained agent demonstrates better behaviors like increasing codebase exploration and more diverse self-verification. These results establish source code as a scalable foundation for constructing RL environments that improve coding agents across diverse software tasks.