BM25在规模化下胜出：检索增强生成范式的规模研究

摘要

检索增强生成（RAG）涵盖词法检索与稠密检索、基于图的索引以及智能体式搜索，但这些范式通常在不同基准上以单一语料库规模进行评估，导致其准确率与成本之间的权衡关系尚不清晰。为弥补这一空白，我们开展了一项受控研究，将语料库规模沿28个严格嵌套的层级进行变化，跨度约为450倍，同时保持查询问题以及一组固定的相关文档和对抗性文档不变。在统一的阅读器模型和统一的评判协议下，我们测定了官方准确率、构建与查询令牌数以及延迟。结果表明，存在一个随规模变化的交叉点，而非某种无条件的优胜方案。文件系统智能体在最小的共享层级上表现最佳，但其顺序式探索在基础语料库上消耗的查询令牌数高达39倍，且随着搜索空间的增大其有效性逐渐下降。在约1000万语料库令牌处，BM25超越该智能体，并在所有更大的共享层级上保持领先，在完整规模下优势接近20个百分点。BM25同时在没有基于LLM的构建的情况下锚定了帕累托前沿的低成本端点。稠密检索仍然高效但准确率较低，而基于图的RAG在达到部署规模之前便遭遇构建瓶颈，其可扩展变体在共享层级上仍落后于BM25。总体而言，语料库规模的增长日益有利于全局候选排序：词法检索是最强的可扩展默认方案，而智能体推理在排序发现之后使用效果最佳，而非替代排序发现。

English

Retrieval-augmented generation (RAG) spans lexical and dense retrieval, graph-based indexing, and agentic search, but these paradigms are usually evaluated on different benchmarks at one corpus size, leaving their accuracy-cost scaling unclear. To bridge this gap, we present a controlled study that varies corpus size along 28 strictly nested tiers spanning roughly 450-fold, while holding questions and a fixed bedrock of relevant and adversarial documents unchanged. Under one reader model and one judging protocol, we measure official accuracy, construction and query tokens, and latency. The results reveal a scale-dependent crossover rather than an unconditional winner. File-System Agent leads at the smallest shared tiers, but its sequential exploration costs 39 times more query tokens at the bedrock and becomes less effective as the search space grows. Around 10 million corpus tokens, BM25 overtakes it and leads at every larger shared tier, with a margin approaching 20 points at full scale. BM25 also anchors the low-cost end of the Pareto frontier without LLM-based construction. Dense retrieval remains efficient but less accurate, whereas graph-based RAG encounters construction walls before deployment scale and its scalable variants remain below BM25 at shared tiers. Overall, corpus growth increasingly favors global candidate ranking: lexical retrieval is the strongest scalable default, while agentic reasoning works best after ranked discovery rather than in place of it.