SKILLER：小语言模型中可复用技能提取的语言级强化学习

摘要

智能体技能是将程序性知识与领域专长封装起来的标准化格式，在智能体框架系统中充当持续约束语言模型行为空间的关键机制，以实现可重复、高质量的任务执行。然而，由于强闭源模型推理成本高昂，当前流行的智能体框架（如Codex和OpenClaw）在部署这些技能以完成现实任务时，成本仍然高得令人望而却步。可在消费级GPU上部署的开源模型能力快速提升，为通过利用基于技能的行为约束大幅降低这些成本提供了极具吸引力的契机。尽管如此，针对这类小型模型自动生成行之有效的专属技能仍是一项重大的实践挑战。为此，我们提出SKILLER——一种自然语言驱动的强化学习框架，旨在为小型模型自动生成执行者专属技能。该框架将强模型作为行动者和评论家，将小型模型智能体系统视为环境，并完全通过自然语言传递所有强化学习信号。基于Qwen3.5-9B和Qwen3.5-4B在五个相关基准上的大量实验评估表明，SKILLER优于三种开源和一种闭源的技能生成或演化方法，在9B模型上取得了4.3至20.4个百分点的绝对提升，在4B模型上取得了1.8至13.3个百分点的绝对提升，同时在SkillsBench的单技能任务中表现卓越，与强闭源模型性能相当。项目代码可在 https://github.com/DANG-ai/SKILLER 获取。

English

Agent skills represent a standardized format for packaging procedural knowledge and domain expertise, serving within agent harness systems as an essential mechanism to continually constrain a language model's behavior space for repeatable, high-quality task execution. However, because strong closed-source models entail high inference costs, current popular agent harnesses, such as Codex and OpenClaw, remain prohibitively expensive when deploying these skills to accomplish real-world tasks. The rapid capability enhancement of open-source models deployable on consumer-grade GPUs presents a compelling opportunity to drastically reduce these costs by leveraging skill-based behavioral constraints. Nevertheless, automatically generating effective skills tailored specifically for such compact models remains a significant practical challenge. To address this, we propose SKILLER, a natural-language-driven reinforcement learning framework designed to automatically generate executor-specific skills for small models, which employs a strong model as the actor and critic, treats the small-model agent system as the environment, and propagates all reinforcement learning signals entirely via natural language. Extensive experimental evaluations across five relevant benchmarks using Qwen3.5-9B and Qwen3.5-4B demonstrate that SKILLER outperforms three open-source and one closed-source skill generation or evolution methods, achieving absolute gains ranging from 4.3 to 20.4 percentage points for the 9B model and 1.8 to 13.3 points for the 4B model, while remarkably matching the performance of strong closed-source models on single-skill tasks in SkillsBench. The project is available at https://github.com/DANG-ai/SKILLER.