BM25 在大規模情境下勝出：檢索增強生成範式的規模化研究

摘要

檢索增強生成（RAG）涵蓋詞彙檢索、稠密檢索、基於圖的索引與代理式搜尋，但這些範式通常在不同基準上以單一語料庫規模進行評估，導致其準確度-成本擴展特性不明。為填補此差距，我們提出一項受控研究，沿著 28 個嚴格嵌套的層級變動語料庫規模，涵蓋約 450 倍的範圍，同時保持問題以及一組固定的相關與對抗性文件（bedrock）不變。在單一閱讀器模型與單一評判協議下，我們量測官方準確度、建構與查詢 token，以及延遲。結果顯示存在依賴規模的交叉，而非無條件的勝者。檔案系統代理在最小的共享層級領先，但其序列式探索在 bedrock 上耗費 39 倍的查詢 token，且隨著搜尋空間擴大而效果下降。在約 1,000 萬個語料庫 token 附近，BM25 超越該代理，並在之後每個更大的共享層級都保持領先，於完整規模時差距接近 20 個百分點。BM25 也無需基於 LLM 的建構，即錨定帕累托前沿的低成本端。稠密檢索仍屬高效但準確度較低，而基於圖的 RAG 在部署規模前便遭遇建構瓶頸，其可擴展變體在共享層級上仍低於 BM25。整體而言，語料庫成長日益有利於全局候選排序：詞彙檢索是最強的可擴展預設，而代理式推理在排序發現之後運作最佳，而非取代排序發現。

English

Retrieval-augmented generation (RAG) spans lexical and dense retrieval, graph-based indexing, and agentic search, but these paradigms are usually evaluated on different benchmarks at one corpus size, leaving their accuracy-cost scaling unclear. To bridge this gap, we present a controlled study that varies corpus size along 28 strictly nested tiers spanning roughly 450-fold, while holding questions and a fixed bedrock of relevant and adversarial documents unchanged. Under one reader model and one judging protocol, we measure official accuracy, construction and query tokens, and latency. The results reveal a scale-dependent crossover rather than an unconditional winner. File-System Agent leads at the smallest shared tiers, but its sequential exploration costs 39 times more query tokens at the bedrock and becomes less effective as the search space grows. Around 10 million corpus tokens, BM25 overtakes it and leads at every larger shared tier, with a margin approaching 20 points at full scale. BM25 also anchors the low-cost end of the Pareto frontier without LLM-based construction. Dense retrieval remains efficient but less accurate, whereas graph-based RAG encounters construction walls before deployment scale and its scalable variants remain below BM25 at shared tiers. Overall, corpus growth increasingly favors global candidate ranking: lexical retrieval is the strongest scalable default, while agentic reasoning works best after ranked discovery rather than in place of it.