RESOURCE2SKILL：从人类创建的多模态资源中提炼可执行的智能体技能

RESOURCE2SKILL: Distilling Executable Agent Skills from Human-Created Multimodal Resources

July 16, 2026

作者: Yijia Fan, Zonglin Di, Zimo Wen, Yifan Yang, Mingxi Cheng, Qi Dai, Bei Liu, Kai Qiu, Yue Dong, Ji Li, Chong Luo

cs.AI

摘要

技能是软件智能体的一种有用抽象，它将人类和智能体的经验转化为可复用的程序化知识。然而，现有的技能库大多是手工编写的、以文本为中心的，或从智能体轨迹中提取的，这使得教程视频和其他多模态人类资源在很大程度上未被充分利用。我们提出了RESOURCE2SKILL框架，该框架将包括教程视频、代码仓库、文章和参考作品在内的多模态资源提炼为软件智能体可执行的技能。RESOURCE2SKILL将这些技能组织成一个层级化的多模态技能维基，其中每个条目结合了结构化文本、代码、视觉示例、元数据和来源信息。这种设计保留了来自不同资源的互补信号：视频捕捉时间顺序操作和视觉效果，代码捕捉可执行的工具模式，而文章或作品则提供概念和风格上的基础。在推理时，智能体从维基中检索并组合相关技能；当覆盖不足时，相同的构建操作符可以在线获取新技能。在七个实际的创作领域中，与无技能智能体相比，RESOURCE2SKILL的平均总得分提高了11.9个百分点，并在28个主聚合模型-领域单元中的26个中优于强大的工具基线。消融实验证实了多模态技能格式、层级化组织、来源多样性、选择策略和在线获取的价值。

English

Skills are a useful abstraction for software agents, turning human and agent experience into reusable procedural knowledge. Yet existing skill libraries are mostly hand-written, text-centric, or derived from agent traces, leaving tutorial videos and other multimodal human resources largely underused. We present RESOURCE2SKILL, a framework that distills multimodal resources, including tutorial videos, repositories, articles, and reference artifacts, into executable skills for software agents. RESOURCE2SKILL organizes these skills as a hierarchical multimodal Skill Wiki, where each entry combines structured text, code, visual examples, metadata, and provenance. This design preserves complementary signals from different resources: videos capture temporal operations and visual effects, code captures executable tool patterns, and articles or artifacts provide conceptual and stylistic grounding. At inference time, agents retrieve and compose relevant skills from the wiki; when coverage is insufficient, the same construction operator can acquire new skills online. Across seven practical authoring domains, RESOURCE2SKILL improves average overall score by +11.9 percentage points over no-skill agents and outperforms strong harness baselines in 26 of 28 main-aggregate model-domain cells. Ablations confirm the value of multimodal skill format, hierarchical organization, source diversity, selection strategy, and online acquisition.