基于Hugging Face知识图谱的推荐、分类与追踪基准测试

Benchmarking Recommendation, Classification, and Tracing Based on Hugging Face Knowledge Graph

May 23, 2025

作者: Qiaosheng Chen, Kaijia Huang, Xiao Zhou, Weiqing Luo, Yuanning Cui, Gong Cheng

cs.AI

摘要

开源机器学习（ML）资源（如模型和数据集）的快速增长加速了信息检索（IR）研究。然而，现有平台如Hugging Face并未明确利用结构化表示，这限制了高级查询和分析，例如追踪模型演变和推荐相关数据集。为填补这一空白，我们构建了HuggingKG，这是首个基于Hugging Face社区构建的大规模知识图谱，用于ML资源管理。HuggingKG拥有260万个节点和620万条边，捕捉了领域特定的关系及丰富的文本属性。在此基础上，我们进一步推出了HuggingBench，一个包含三个新颖测试集合的多任务基准，用于资源推荐、分类和追踪等IR任务。实验揭示了HuggingKG及其衍生任务的独特特性。这两项资源均已公开，有望推动开源资源共享与管理领域的研究进展。

English

The rapid growth of open source machine learning (ML) resources, such as models and datasets, has accelerated IR research. However, existing platforms like Hugging Face do not explicitly utilize structured representations, limiting advanced queries and analyses such as tracing model evolution and recommending relevant datasets. To fill the gap, we construct HuggingKG, the first large-scale knowledge graph built from the Hugging Face community for ML resource management. With 2.6 million nodes and 6.2 million edges, HuggingKG captures domain-specific relations and rich textual attributes. It enables us to further present HuggingBench, a multi-task benchmark with three novel test collections for IR tasks including resource recommendation, classification, and tracing. Our experiments reveal unique characteristics of HuggingKG and the derived tasks. Both resources are publicly available, expected to advance research in open source resource sharing and management.