WildGraphBench：基於野生來源語料庫的GraphRAG基準測試

摘要

基於圖譜的檢索增強生成（GraphRAG）通過將外部知識組織為層次化圖結構，實現了跨多篇文檔的分散證據高效檢索與聚合。然而，現有許多GraphRAG基準測試僅使用簡短的精選段落作為外部知識，未能充分評估系統在處理長上下文和大規模異構文檔的真實場景中的表現。為填補這一空白，我們推出了WildGraphBench基準測試，專門用於評估GraphRAG在真實環境下的性能。我們利用維基百科的獨特結構——其連貫敘述均基於長篇異構的外部參考文檔——構建了反映真實場景的基準。具體而言，我們採樣了12個頂級主題的文章，將其外部參考文獻作為檢索語料庫，並將引文鏈接的陳述作為事實依據，最終生成涵蓋三種複雜度層級的1,100個問題：單事實問答、多事實問答和章節級摘要。在多個基線模型上的實驗表明，當證據來源數量適中時，當前GraphRAG流程有助於多事實聚合，但這種聚合範式可能過度強調高層次陳述而忽略細粒度細節，導致在摘要任務中表現較弱。項目頁面：https://github.com/BstWPY/WildGraphBench。

English

Graph-based Retrieval-Augmented Generation (GraphRAG) organizes external knowledge as a hierarchical graph, enabling efficient retrieval and aggregation of scattered evidence across multiple documents. However, many existing benchmarks for GraphRAG rely on short, curated passages as external knowledge, failing to adequately evaluate systems in realistic settings involving long contexts and large-scale heterogeneous documents. To bridge this gap, we introduce WildGraphBench, a benchmark designed to assess GraphRAG performance in the wild. We leverage Wikipedia's unique structure, where cohesive narratives are grounded in long and heterogeneous external reference documents, to construct a benchmark reflecting real-word scenarios. Specifically, we sample articles across 12 top-level topics, using their external references as the retrieval corpus and citation-linked statements as ground truth, resulting in 1,100 questions spanning three levels of complexity: single-fact QA, multi-fact QA, and section-level summarization. Experiments across multiple baselines reveal that current GraphRAG pipelines help on multi-fact aggregation when evidence comes from a moderate number of sources, but this aggregation paradigm may overemphasize high-level statements at the expense of fine-grained details, leading to weaker performance on summarization tasks. Project page:https://github.com/BstWPY/WildGraphBench.

WildGraphBench：基於野生來源語料庫的GraphRAG基準測試

WildGraphBench: Benchmarking GraphRAG with Wild-Source Corpora

摘要

Support