WildGraphBench：基于野生源语料库的GraphRAG基准测试

摘要

基于图谱的检索增强生成（GraphRAG）通过将外部知识组织为层次化图谱，实现了跨多文档分散证据的高效检索与聚合。然而，现有GraphRAG基准测试多采用经过整理的短文本片段作为外部知识，难以充分评估系统在长上下文和大规模异构文档的真实场景中的表现。为弥补这一不足，我们推出了WildGraphBench基准测试，旨在评估实际应用场景下的GraphRAG性能。我们利用维基百科的独特结构——其连贯叙述均源自长篇异构的外部参考文献——构建了反映真实场景的基准。具体而言，我们选取12个顶级主题领域的文章，以其外部参考文献作为检索语料库，将引文关联的陈述作为标准答案，最终构建包含1,100个问题的数据集，涵盖三个复杂度层级：单事实问答、多事实问答和章节级摘要。多基线实验表明，当证据来源数量适中时，当前GraphRAG流程有助于多事实聚合，但这种聚合范式可能过度强调高层级陈述而忽略细粒度细节，导致在摘要任务中表现较弱。项目页面：https://github.com/BstWPY/WildGraphBench。

English

Graph-based Retrieval-Augmented Generation (GraphRAG) organizes external knowledge as a hierarchical graph, enabling efficient retrieval and aggregation of scattered evidence across multiple documents. However, many existing benchmarks for GraphRAG rely on short, curated passages as external knowledge, failing to adequately evaluate systems in realistic settings involving long contexts and large-scale heterogeneous documents. To bridge this gap, we introduce WildGraphBench, a benchmark designed to assess GraphRAG performance in the wild. We leverage Wikipedia's unique structure, where cohesive narratives are grounded in long and heterogeneous external reference documents, to construct a benchmark reflecting real-word scenarios. Specifically, we sample articles across 12 top-level topics, using their external references as the retrieval corpus and citation-linked statements as ground truth, resulting in 1,100 questions spanning three levels of complexity: single-fact QA, multi-fact QA, and section-level summarization. Experiments across multiple baselines reveal that current GraphRAG pipelines help on multi-fact aggregation when evidence comes from a moderate number of sources, but this aggregation paradigm may overemphasize high-level statements at the expense of fine-grained details, leading to weaker performance on summarization tasks. Project page:https://github.com/BstWPY/WildGraphBench.

WildGraphBench：基于野生源语料库的GraphRAG基准测试

WildGraphBench: Benchmarking GraphRAG with Wild-Source Corpora

摘要

Support