RESOURCE2SKILL：從人類創建的多模態資源中提煉可執行的代理技能

RESOURCE2SKILL: Distilling Executable Agent Skills from Human-Created Multimodal Resources

July 16, 2026

作者: Yijia Fan, Zonglin Di, Zimo Wen, Yifan Yang, Mingxi Cheng, Qi Dai, Bei Liu, Kai Qiu, Yue Dong, Ji Li, Chong Luo

cs.AI

摘要

技能是軟體代理程式中一種有用的抽象概念，能將人類與代理的經驗轉化為可重複使用的程序性知識。然而，現有的技能庫大多為手動撰寫、以文字為中心，或衍生自代理軌跡，導致教學影片及其他多模態人類資源未能充分利用。我們提出 RESOURCE2SKILL 框架，該框架將多模態資源（包括教學影片、儲存庫、文章及參考素材）提煉為軟體代理程式可執行的技能。RESOURCE2SKILL 將這些技能組織成一個分層式多模態技能維基，每個條目結合結構化文字、程式碼、視覺範例、後設資料及來源資訊。這種設計能保留來自不同資源的互補訊號：影片捕捉時間序列操作與視覺效果，程式碼捕捉可執行的工具模式，而文章或素材則提供概念與風格基礎。在推論階段，代理程式從維基中檢索並組合相關技能；若覆蓋範圍不足，相同的建構運算子可線上獲取新技能。在七個實際創作文領域中，RESOURCE2SKILL 相較於無技能代理程式，平均總分提升 11.9 個百分點，並在 28 個主要聚合模型-領域組合中的 26 個優於強基線。消融實驗證實了多模態技能格式、分層組織、來源多樣性、選擇策略及線上獲取的價值。

English

Skills are a useful abstraction for software agents, turning human and agent experience into reusable procedural knowledge. Yet existing skill libraries are mostly hand-written, text-centric, or derived from agent traces, leaving tutorial videos and other multimodal human resources largely underused. We present RESOURCE2SKILL, a framework that distills multimodal resources, including tutorial videos, repositories, articles, and reference artifacts, into executable skills for software agents. RESOURCE2SKILL organizes these skills as a hierarchical multimodal Skill Wiki, where each entry combines structured text, code, visual examples, metadata, and provenance. This design preserves complementary signals from different resources: videos capture temporal operations and visual effects, code captures executable tool patterns, and articles or artifacts provide conceptual and stylistic grounding. At inference time, agents retrieve and compose relevant skills from the wiki; when coverage is insufficient, the same construction operator can acquire new skills online. Across seven practical authoring domains, RESOURCE2SKILL improves average overall score by +11.9 percentage points over no-skill agents and outperforms strong harness baselines in 26 of 28 main-aggregate model-domain cells. Ablations confirm the value of multimodal skill format, hierarchical organization, source diversity, selection strategy, and online acquisition.