可执行知识图谱在AI研究复现中的应用

摘要

复制人工智能研究对于大型语言模型（LLM）代理而言是一项关键却充满挑战的任务。现有方法往往难以生成可执行代码，主要归因于背景知识的不足以及检索增强生成（RAG）方法的局限性，后者未能捕捉到参考文献中隐含的技术细节。此外，先前的方法常常忽视了宝贵的实现层面代码信号，并缺乏支持多粒度检索与重用的结构化知识表示。为应对这些挑战，我们提出了可执行知识图谱（xKG），这是一个模块化且可插拔的知识库，能自动整合从科学文献中提取的技术见解、代码片段及领域特定知识。当xKG被集成到采用两种不同LLM的三种代理框架中时，在PaperBench上展现了显著的性能提升（使用o3-mini时提升10.9%），证明了其作为自动化AI研究复制的通用且可扩展解决方案的有效性。代码将在https://github.com/zjunlp/xKG 发布。

English

Replicating AI research is a crucial yet challenging task for large language model (LLM) agents. Existing approaches often struggle to generate executable code, primarily due to insufficient background knowledge and the limitations of retrieval-augmented generation (RAG) methods, which fail to capture latent technical details hidden in referenced papers. Furthermore, previous approaches tend to overlook valuable implementation-level code signals and lack structured knowledge representations that support multi-granular retrieval and reuse. To overcome these challenges, we propose Executable Knowledge Graphs (xKG), a modular and pluggable knowledge base that automatically integrates technical insights, code snippets, and domain-specific knowledge extracted from scientific literature. When integrated into three agent frameworks with two different LLMs, xKG shows substantial performance gains (10.9% with o3-mini) on PaperBench, demonstrating its effectiveness as a general and extensible solution for automated AI research replication. Code will released at https://github.com/zjunlp/xKG.

可执行知识图谱在AI研究复现中的应用

Executable Knowledge Graphs for Replicating AI Research

摘要

Support