揭秘智能体技能：何以奏效，直至失效

摘要

技能已成为一种实用且有效的方法，通过在推理阶段提供结构化知识包来增强LLM智能体的能力。然而，现有评估大多仅关注技能是否提升聚合任务成功率，忽略了一个更根本的问题：**技能在何时起作用、为何起作用、又在何处失效？**通过在多种基准测试、智能体框架和LLM上进行受控实验，我们分离了表示方式、结果标注、检索难度和跨框架鲁棒性对技能的影响。为更深入地回答该问题，我们设计了一项对比研究，将受控定量实验与配对轨迹分析相结合。我们对受控实验中的8,135条试验记录进行了规范化处理，并从240条开放式编码记录中保留了238个有效唯一标签。我们将这些观察结果整合为一个包含三个高层类别和十二种技能使用模式的分类体系：当嘈杂轨迹转化为稳定执行的程序性锚点时，技能便发挥作用。在匹配比较中，技能相比工作流记忆提升6.06个百分点。程序性锚定占技能应用案例的65.7%，而显式知识注入仅占4.5%，表明技能的作用在于稳定行为，而非注入缺失的事实。检索是另一个独立瓶颈：随着候选池规模从5增至100，实际使用精确率从29.6%降至3.3%。易混淆的干扰项虽损害离线识别效果，但下游成功率保持稳定；精确的真实标注调用既非充分条件，也非必要条件。技能在假设过于僵化、上下文不兼容或适应不足的情况下会失效。这些发现将评估从聚合成功率推进到更深入的层面，并为可靠的自进化智能体提供指导。

English

Skills have emerged as a practical and effective approach for enhancing LLM agents at inference time through structured packages of knowledge. However, existing evaluations largely measure whether skills improve aggregated task success, leaving a more fundamental question underexplored: \textbf{When do skills help, why do they work, and where do they fail?} Through controlled experiments across various benchmarks, agent harnesses and LLMs, we isolate the effects of representation, outcome annotation, retrieval difficulty, and cross-framework robustness of skills. To further answer this question, we design a contrastive study that combines controlled quantitative experiments with paired trajectory analysis. We normalize 8,135 trial records from controlled experiments and retain 238 valid unique labels from 240 open-coded records. We consolidate these observations into a taxonomy of three high-level categories and twelve skill-use modes: skills work when noisy trajectories become procedural anchors that stabilize execution. Skills improve over Workflow Memory by 6.06 points in matched comparisons. Procedural anchoring accounts for 65.7\% of skill cases, versus 4.5\% for explicit knowledge injection, showing that skills stabilize action rather than inject missing facts. Retrieval is a separate bottleneck: as pools grow from 5 to 100, actual-use precision falls from 29.6\% to 3.3\%. Confusable distractors impair offline identification, yet downstream success remains stable; exact ground-truth invocation is neither sufficient nor necessary. Skills fail under brittle assumptions, incompatible contexts, or insufficient adaptation. These findings move evaluation beyond aggregate success rates and guide reliable self-evolving agents.