CodeMidas：从代码自身出发扩展智能体编程强化学习环境

CodeMidas: Scaling Agentic Coding RL Environments from Code Itself

September 18, 2026

作者: Bowen Ye, Lei Li, Shicheng Li, Zihao Yue, Linghao Zhang, Hanglong Lv, Yuanxin Liu, Wenhan Ma, Hao Tian, Rang Li, Jinhao Dong, Yikai Zhao, Xiangwei Deng, Hailin Zhang, Liang Zhao, Qi Liu, Lingpeng Kong, Tong Yang, Fuli Luo

cs.AI

摘要

通过强化学习（RL）训练能力强的编码智能体，需要具备可靠验证器的多样化任务。开源代码库是此类任务的丰富来源，而现有方法通常依赖 issue 和 commit 等开发产物，限制了可提取任务的范围。为了更好地扩展 RL 环境，我们提出 CodeMidas，一种智能体流水线，仅以源代码作为任务特定输入，将现有代码库中已实现的功能转化为可执行的 RL 环境。CodeMidas 将智能体计算资源分配到环境构建的每个阶段：智能体探索已实现功能以制定行为规范，基于原始代码的执行构建测试，并通过执行检查与重复的解法 rollout 来验证和筛选候选任务。所得数据集包含来自 3,185 个开源代码库的 5,545 个训练任务，涵盖 23 种编程语言和 15 个技术领域。使用 GRPO 在这些任务上训练 MiMo-V2.5，可在五个多样化基准上全面提升性能，覆盖问题修复（DeepSWE +11.7%）、整体程序构建（ProgramBench +17%）和终端任务（Terminal-Bench v2.1 +8.5%）。消融实验表明，增加高质量训练任务的数量可提升性能。轨迹分析显示，经 RL 训练的智能体展现出更好的行为，例如增加代码库探索以及更多样化的自我验证。这些结果确立了源代码作为可扩展基础的地位，可用于构建提升编码智能体在多样化软件任务中表现的 RL 环境。

English

Training capable coding agents via reinforcement learning (RL) requires diverse tasks with reliable verifiers. Open-source codebases offer a rich source of such tasks, while existing methods typically rely on development artifacts such as issues and commits, limiting the range of tasks that can be extracted. To better scale RL environments, we present CodeMidas, an agentic pipeline that turns implemented functionality in existing codebases into executable RL environments using source code as its only task-specific input. CodeMidas allocates agentic compute to every stage of environment construction: agents explore implemented functionality to formulate behavioral specifications, construct tests grounded in execution of the original code, and validate and filter candidate tasks through execution checks and repeated solution rollouts. The resulting dataset has 5,545 training tasks from 3,185 open-source codebases spanning 23 programming languages and 15 technical domains. Training MiMo-V2.5 on these tasks with GRPO improves performance on all five diverse benchmarks, covering issue repair (DeepSWE + 11.7%), whole-program construction (ProgramBench +17%), and terminal work (Terminal-Bench v2.1 +8.5%). Ablations show that increasing the number of high-quality training tasks improves performance. Trajectory analysis shows the RL-trained agent demonstrates better behaviors like increasing codebase exploration and more diverse self-verification. These results establish source code as a scalable foundation for constructing RL environments that improve coding agents across diverse software tasks.