ChartWalker：跨圖表RAG任務的基準測試

摘要

跨圖表檢索增強生成（RAG）對於科學、商業與政治領域中複雜的多模態分析任務至關重要。然而，現有的基準測試要麼專注於結構化且文本化的表格，要麼僅透過擷取關鍵點來生成跨圖表問題，這往往導致查詢與證據之間出現詞彙重疊，並產生邏輯不一致的推理鏈。為解決此問題，我們提出ChartWalker，一個用於構建具挑戰性的跨圖表RAG任務的新穎框架。ChartWalker採用了專為圖表設計的層次化知識圖譜構建方法，按粒度組織實體與關係，以保留分析結構。接著，我們提出一種結構感知採樣演算法，能夠合成語義連貫的多跳推理路徑，從而實現對問答生成中查詢難度與粒度的明確控制。基於此框架，我們釋出了ChartWalker-Bench，這是一個涵蓋多樣領域與各類跨圖表查詢類型的全面基準測試。針對主要RAG範式的廣泛評估揭示了顯著的性能差距，凸顯了該基準測試的難度與實用性。此外，我們還提供了ChartWalker-Agent，作為一個智能體基線，以促進分析並啟發未來的系統設計。

English

Cross-Chart Retrieval-Augmented Generation (RAG) is critical for complex multi-modal analytical tasks in scientific, business, and political domains. However, existing benchmarks either focus on tables, which are well-structured and textualized, or generate cross-chart questions by simply extracting key points, which often induces lexical overlap between queries and evidence and yields logically inconsistent reasoning chains. To address this, we introduce ChartWalker, a novel framework for constructing challenging cross-chart RAG tasks. ChartWalker features a hierarchical knowledge graph construction method tailored to charts, which organizes entities and relations by granularity to preserve analytical structure. We then propose a structure-aware sampling algorithm that synthesizes semantically coherent, multi-hop reasoning paths, enabling explicit control over query difficulty and granularity for QA generation. Built with this framework, we release ChartWalker-Bench, a comprehensive benchmark spanning diverse domains and cross-chart query types. Extensive evaluations across major RAG paradigms reveal significant performance gaps, underscoring the benchmark's difficulty and utility. Furthermore, we provide ChartWalker-Agent, an agentic baseline to facilitate analysis and inspire future system design.