可執行知識圖譜於人工智慧研究複現之應用

摘要

複製人工智慧研究對於大型語言模型（LLM）代理而言是一項至關重要卻又充滿挑戰的任務。現有方法往往難以生成可執行的程式碼，主要原因在於背景知識的不足以及檢索增強生成（RAG）方法的侷限性，這些方法未能捕捉到參考文獻中隱藏的技術細節。此外，先前的方法傾向於忽略有價值的實作層面程式碼信號，並缺乏支持多粒度檢索與重用的結構化知識表示。為克服這些挑戰，我們提出了可執行知識圖譜（xKG），這是一個模組化且可插拔的知識庫，能自動整合從科學文獻中提取的技術見解、程式碼片段及領域特定知識。當xKG被整合到採用兩種不同LLM的三種代理框架中時，在PaperBench上展現了顯著的性能提升（使用o3-mini時提升10.9%），證明了其作為自動化AI研究複製的通用且可擴展解決方案的有效性。程式碼將於https://github.com/zjunlp/xKG發布。

English

Replicating AI research is a crucial yet challenging task for large language model (LLM) agents. Existing approaches often struggle to generate executable code, primarily due to insufficient background knowledge and the limitations of retrieval-augmented generation (RAG) methods, which fail to capture latent technical details hidden in referenced papers. Furthermore, previous approaches tend to overlook valuable implementation-level code signals and lack structured knowledge representations that support multi-granular retrieval and reuse. To overcome these challenges, we propose Executable Knowledge Graphs (xKG), a modular and pluggable knowledge base that automatically integrates technical insights, code snippets, and domain-specific knowledge extracted from scientific literature. When integrated into three agent frameworks with two different LLMs, xKG shows substantial performance gains (10.9% with o3-mini) on PaperBench, demonstrating its effectiveness as a general and extensible solution for automated AI research replication. Code will released at https://github.com/zjunlp/xKG.

可執行知識圖譜於人工智慧研究複現之應用

Executable Knowledge Graphs for Replicating AI Research

摘要

Support