SKILLER：小型語言模型中基於語言層級強化學習的可重用技能提取

摘要

智慧體技能代表了一種用於封裝程序性知識與領域專業知識的標準化格式，在智慧體框架系統中扮演關鍵機制，持續約束語言模型的行為空間，以實現可重複且高品質的任務執行。然而，由於強大的閉源模型涉及高昂的推論成本，目前主流的智慧體框架（如 Codex 和 OpenClaw）在部署這些技能以完成真實世界任務時，成本仍然過高而難以普遍採用。可在消費級 GPU 上運行的開源模型能力快速提升，為透過基於技能的行為約束大幅降低這些成本提供了極具吸引力的契機。儘管如此，針對此類輕量模型自動生成有效技能仍是一項重大的實際挑戰。為應對此問題，我們提出 SKILLER——一個以自然語言驅動的強化學習框架，專為小型模型自動生成執行者特定技能。該框架以強大模型作為行動者與評論家，將小型模型智慧體系統視為環境，並完全透過自然語言傳遞所有強化學習訊號。在五個相關基準測試上使用 Qwen3.5-9B 與 Qwen3.5-4B 進行的大規模實驗評估顯示，SKILLER 優於三種開源與一種閉源的技能生成或演化方法，在 9B 模型上取得 4.3 至 20.4 個百分點的絕對提升，在 4B 模型上取得 1.8 至 13.3 個百分點的提升，同時在 SkillsBench 的單技能任務上顯著匹敵強大閉源模型的效能。該專案可於 https://github.com/DANG-ai/SKILLER 取得。

English

Agent skills represent a standardized format for packaging procedural knowledge and domain expertise, serving within agent harness systems as an essential mechanism to continually constrain a language model's behavior space for repeatable, high-quality task execution. However, because strong closed-source models entail high inference costs, current popular agent harnesses, such as Codex and OpenClaw, remain prohibitively expensive when deploying these skills to accomplish real-world tasks. The rapid capability enhancement of open-source models deployable on consumer-grade GPUs presents a compelling opportunity to drastically reduce these costs by leveraging skill-based behavioral constraints. Nevertheless, automatically generating effective skills tailored specifically for such compact models remains a significant practical challenge. To address this, we propose SKILLER, a natural-language-driven reinforcement learning framework designed to automatically generate executor-specific skills for small models, which employs a strong model as the actor and critic, treats the small-model agent system as the environment, and propagates all reinforcement learning signals entirely via natural language. Extensive experimental evaluations across five relevant benchmarks using Qwen3.5-9B and Qwen3.5-4B demonstrate that SKILLER outperforms three open-source and one closed-source skill generation or evolution methods, achieving absolute gains ranging from 4.3 to 20.4 percentage points for the 9B model and 1.8 to 13.3 points for the 4B model, while remarkably matching the performance of strong closed-source models on single-skill tasks in SkillsBench. The project is available at https://github.com/DANG-ai/SKILLER.