ChatPaper.aiChatPaper

I'd be happy to translate your text into Simplified Chinese. However, you haven't provided the source text yet — you've only given me the title "Environment Evolution for Terminal Agents." Please share the full content you'd like translated, and I'll translate it into Simplified Chinese with accurate technical terminology and academic style.

Environment Evolution for Terminal Agents

September 3, 2026
作者: Zhiyuan Fan, Tinghao Yu, Yuanjun Cai, Jiang Zhou, Jiangtao Guan, Jincheng Liu, Yun Yang, Dingxin Hu, Zhuo Han, Xing Wu, Feng Zhang, Lilin Wang
cs.AI

摘要

扩展交互式和可验证环境对于训练终端智能体至关重要。随着前沿模型能力不断增强,从零开始合成的环境变得不再具有挑战性,因而提供的学习信号也有限。近期提出的共同进化方法基于模型在回滚过程中暴露的弱点,在模型可学习前沿附近迭代合成环境。然而,这类方法对在策略回滚的依赖限制了其泛化能力,并且随着模型能力增强,持续提供学习信号的能力也会受限。本文提出环境进化方法,该方法以离策略方式逐步提升环境难度,并在训练过程中逐代调度进化后的环境,以提供持续的学习信号。我们从多轮学习目标出发推导出三条影响环境难度的进化方向,并通过循环工程化的多智能体框架沿这些方向实施进化。基于Hy4 preview、Claude Opus 5和GPT-5.6 Sol的定量回滚实验表明,环境进化能够持续生成更具难度的环境。我们通过简单的长时程强化学习训练在Qwen3.6-27B和Qwen3.6-35B-A3B上验证了该方法的有效性,分别在Terminal-Bench 2.1基准上将性能提升了14.4和18.0个百分点。
English
Scaling interactive and verifiable environments is critical for training terminal agents. As frontier models become more capable, environments synthesized from scratch become less challenging and thus provide limited learning signals. Recent co-evolution methods iteratively synthesize environments near the model's learnable frontier based on weaknesses exposed during rollouts. However, their dependence on on-policy rollouts limits generalization and the continuous provision of learning signals as the model becomes stronger. In this paper, we propose environment evolution, which incrementally increases environment difficulty off-policy and schedules the evolved environments generation by generation during training to provide continuous learning signals. We derive three evolution directions that influence environment difficulty from the multi-turn learning objective and then implement evolution along these directions through a loop-engineered multi-agent harness. Quantitative rollout experiments with Hy4 preview, Claude Opus 5, and GPT-5.6 Sol show that environment evolution consistently produces more difficult environments. We validate its effectiveness on Qwen3.6-27B and Qwen3.6-35B-A3B through simple long-horizon RL training, improving their performance by 14.4 and 18.0 percentage points on Terminal-Bench 2.1, respectively.