EnvHarness：为智能体学习唤醒静态世界

摘要

LLM代理通过与环境交互进行学习，然而这些环境是手工构建且静态的：它们对代理的弱点视而不见，并随着代理能力的提升迅速被抛在身后。尽管最近的环境生成方法试图解决这一问题，但它们需要领域特定的流水线，依赖昂贵或不可靠的验证器，且生成的仍是静态环境。为减轻从零重建环境的工程负担，我们提出了环境调控框架（EnvHarness）——一个由即插即用组件构成的可编程层，它封装静态环境以重塑其行为，而无需修改底层逻辑。EnvHarness通过标准接口运作，可适用于多种领域，同时确保每个被重塑的环境都保留其原始验证器。为使该过程自动化，我们引入了EnvRigger：它将目标策略视为黑盒，观察其执行轨迹，合成针对已诊断缺陷的EnvHarness组件，并通过全新的展开对其进行验证。在四个领域的五个基准测试中，EnvHarness的表现优于原始环境和领域特定的环境生成流水线，在留出实例上实现了高达9.0分的提升，同时执行步骤减少了9.8%。此外，EnvHarness为强化学习提供了更优的优化信号，使策略与其环境能够持续、有针对性地协同进化。

English

LLM agents learn by interacting with environments, yet these environments are hand-built and static: blind to an agent's weaknesses, and quickly left behind as it improves. While recent environment generation methods attempt to address this, they require domain-specific pipelines, rely on expensive or unreliable verifiers, and still produce static environments. To alleviate the engineering burden of rebuilding environments from scratch, we propose Environment Harness (EnvHarness), a programmable layer of plug-in components that wraps a static environment to reshape its behavior without modifying the underlying logic. Operating through standard interfaces, EnvHarness applies across diverse domains while ensuring every reshaped environment retains its original verifier. To automate this process, we introduce EnvRigger, which treats the target policy as a black box, observing its execution trajectories to synthesize EnvHarness components targeting diagnosed flaws, and validating them via fresh rollouts. Across five benchmarks in four domains, EnvHarness outperforms both original environments and domain-specific environment generation pipelines, achieving up to a 9.0-point improvement on held-out instances with 9.8% fewer execution steps. Furthermore, EnvHarness provides a superior optimization signal for reinforcement learning, enabling continuous, targeted co-evolution of the policy and its environment.