SkillCoach：用於評估與強化自主技能運用的自演化評分量規

摘要

技能正逐漸成為大型語言模型代理的可重複使用操作層，用來編碼標準作業程序、領域規則、工具工作流程、腳本及驗證程序。在實際的技能庫中，重疊的技能使得可靠使用技能變得困難。最終驗證器的成功與否對於評估與訓練而言過於粗糙，因為代理可能透過試誤過程，選取干擾技能、跳過必要步驟、不當組合工作流程或忽略最終檢查，仍能通過驗證。我們提出SkillCoach，這是一個自我演化的評量框架，用於評估與強化代理的技能使用能力。SkillCoach從實際執行歷程中推導出基於技能的流程評量標準，並從四個維度評估軌跡：技能選擇、技能遵循、技能組合，以及基於技能的反思。它將外部驗證器作為獨立結果信號保留，從而區分流程品質與偶然的任務成功。演化後的評量標準更進一步作為流程監督信號，用於篩選高品質的訓練軌跡。實驗結果顯示，演化後的評量標準顯著提升評估品質，揭示出最終準確率無法反映的失敗，並在強化代理技能使用方面，提供比僅基於結果過濾更強的監督信號。

English

Skills are becoming a reusable operational layer for LLM agents, encoding SOPs, domain rules, tool workflows, scripts, and validation routines. In realistic skill repositories, overlapping skills make reliable skill-use difficult. Final verifier success is too coarse for both evaluation and training, since an agent may pass through trial and error while selecting distractor skills, skipping required steps, composing workflows incorrectly or omitting final checks. We introduce SkillCoach, a self-evolving rubric framework for evaluating and enhancing agentic skill-use. SkillCoach derives skill-grounded process rubrics from real rollouts and evaluates trajectories along four dimensions: skill selection, skill following, skill composition, and skill-grounded reflection. It keeps the external verifier as a separate outcome signal, allowing process quality to be distinguished from accidental task success. The evolved rubrics further serve as process supervision for selecting high-quality training trajectories. Experiments show that evolved rubrics substantially improve evaluation quality, expose failures hidden by final accuracy, and provide stronger supervision signals than outcome-only filtering for enhancing agentic skill-use.