SkillCoach：自我进化的评分标准，用于评估与增强代理技能运用

摘要

技能正逐步成为大型语言模型智能体的可重用操作层，涵盖标准作业程序、领域规则、工具工作流、脚本及验证流程。在实际技能库中，技能重叠导致可靠调用变得困难。最终验证器的通过状态既不适合评估也不适合训练——因为智能体可能通过试错方式使用干扰技能、跳过必要步骤、错误编排工作流或遗漏最终检查。为此，我们提出SkillCoach框架——一种面向评估与优化智能体技能调用的自演进评估准则体系。SkillCoach从真实执行轨迹中提取基于技能的过程评估准则，沿四个维度评估轨迹质量：技能选择、技能遵循、技能编排及基于技能的反思。该框架将外部验证器作为独立的结果信号，使过程质量得以与偶然任务成功相区分。演进后的评估准则进一步作为过程监督信号，用于筛选高质量训练轨迹。实验证明，演进准则显著提升评估质量，能够揭露最终准确率所掩盖的失败模式，并为优化智能体技能调用提供比纯结果过滤更强的监督信号。

English

Skills are becoming a reusable operational layer for LLM agents, encoding SOPs, domain rules, tool workflows, scripts, and validation routines. In realistic skill repositories, overlapping skills make reliable skill-use difficult. Final verifier success is too coarse for both evaluation and training, since an agent may pass through trial and error while selecting distractor skills, skipping required steps, composing workflows incorrectly or omitting final checks. We introduce SkillCoach, a self-evolving rubric framework for evaluating and enhancing agentic skill-use. SkillCoach derives skill-grounded process rubrics from real rollouts and evaluates trajectories along four dimensions: skill selection, skill following, skill composition, and skill-grounded reflection. It keeps the external verifier as a separate outcome signal, allowing process quality to be distinguished from accidental task success. The evolved rubrics further serve as process supervision for selecting high-quality training trajectories. Experiments show that evolved rubrics substantially improve evaluation quality, expose failures hidden by final accuracy, and provide stronger supervision signals than outcome-only filtering for enhancing agentic skill-use.