解密代理技能：為何它們有效——直到失效為止

摘要

技能已成為一種實用且有效的方法，可在推理時透過結構化的知識包來增強大型語言模型智慧體。然而，現有評估大多僅衡量技能是否提升整體任務成功率，卻未深入探討一個更根本的問題：**技能何時有幫助、為何有效、又在何處失效？** 透過在各種基準、智慧體框架與大型語言模型上進行的受控實驗，我們分離出表徵、結果標註、檢索難度及技能跨框架穩健性的個別效應。為進一步回答此問題，我們設計了一項對比研究，結合受控量化實驗與成對軌跡分析。我們將受控實驗中的 8,135 筆試驗紀錄標準化，並從 240 筆開放編碼紀錄中保留 238 個有效唯一標籤。我們將這些觀察結果歸納為一個涵蓋三個高層級類別與十二種技能使用模式的分類法：當嘈雜軌跡轉變為能穩定執行的程序性錨點時，技能即發揮效用。在匹配比較中，技能較工作流記憶提升 6.06 分。程序性錨定佔技能案例的 65.7%，而顯式知識注入僅佔 4.5%，顯示技能的作用在於穩定行動，而非注入缺失的事實。檢索是另一個獨立瓶頸：當候選池從 5 個擴增至 100 個時，實際使用精確度從 29.6% 下降至 3.3%。易混淆的干擾項會損害離線識別，但下游成功率仍保持穩定；精確呼叫真實標籤既非充分條件也非必要條件。技能在假設脆弱、情境不相容或適應不足時會失效。這些發現將評估推昇至整體成功率之上，並為可靠的自演化智慧體提供指引。

English

Skills have emerged as a practical and effective approach for enhancing LLM agents at inference time through structured packages of knowledge. However, existing evaluations largely measure whether skills improve aggregated task success, leaving a more fundamental question underexplored: \textbf{When do skills help, why do they work, and where do they fail?} Through controlled experiments across various benchmarks, agent harnesses and LLMs, we isolate the effects of representation, outcome annotation, retrieval difficulty, and cross-framework robustness of skills. To further answer this question, we design a contrastive study that combines controlled quantitative experiments with paired trajectory analysis. We normalize 8,135 trial records from controlled experiments and retain 238 valid unique labels from 240 open-coded records. We consolidate these observations into a taxonomy of three high-level categories and twelve skill-use modes: skills work when noisy trajectories become procedural anchors that stabilize execution. Skills improve over Workflow Memory by 6.06 points in matched comparisons. Procedural anchoring accounts for 65.7\% of skill cases, versus 4.5\% for explicit knowledge injection, showing that skills stabilize action rather than inject missing facts. Retrieval is a separate bottleneck: as pools grow from 5 to 100, actual-use precision falls from 29.6\% to 3.3\%. Confusable distractors impair offline identification, yet downstream success remains stable; exact ground-truth invocation is neither sufficient nor necessary. Skills fail under brittle assumptions, incompatible contexts, or insufficient adaptation. These findings move evaluation beyond aggregate success rates and guide reliable self-evolving agents.