DoTA-RAG：动态思维聚合检索增强生成

摘要

本文介绍了DoTA-RAG（动态思维聚合检索增强生成系统），这是一个针对高吞吐量、大规模网络知识索引优化的检索增强生成系统。传统的RAG管道在处理海量、多样化数据集时，常面临高延迟和准确性受限的问题。DoTA-RAG通过三阶段管道应对这些挑战：查询重写、动态路由至专业子索引，以及多阶段检索与排序。我们进一步通过评估并选择更优的嵌入模型，对FineWeb-10BT大规模语料库进行重新嵌入，从而提升了检索效果。此外，我们利用DataMorgana框架生成了涵盖广泛WebOrganizer主题和格式的500个问答数据集，丰富了测试资源。DoTA-RAG将答案正确率从基线（使用LiveRAG预构建向量存储）的0.752提升至1.478，同时保持了低延迟，并在Live Challenge Day上取得了0.929的正确率。这些成果彰显了DoTA-RAG在需要快速、可靠访问大规模动态知识源领域中的实际部署潜力。

English

In this paper, we introduce DoTA-RAG (Dynamic-of-Thought Aggregation RAG), a retrieval-augmented generation system optimized for high-throughput, large-scale web knowledge indexes. Traditional RAG pipelines often suffer from high latency and limited accuracy over massive, diverse datasets. DoTA-RAG addresses these challenges with a three-stage pipeline: query rewriting, dynamic routing to specialized sub-indexes, and multi-stage retrieval and ranking. We further enhance retrieval by evaluating and selecting a superior embedding model, re-embedding the large FineWeb-10BT corpus. Moreover, we create a diverse Q&A dataset of 500 questions generated via the DataMorgana setup across a broad range of WebOrganizer topics and formats. DoTA-RAG improves the answer correctness score from 0.752 (baseline, using LiveRAG pre-built vector store) to 1.478 while maintaining low latency, and it achieves a 0.929 correctness score on the Live Challenge Day. These results highlight DoTA-RAG's potential for practical deployment in domains requiring fast, reliable access to large and evolving knowledge sources.

DoTA-RAG：动态思维聚合检索增强生成

DoTA-RAG: Dynamic of Thought Aggregation RAG

摘要

Support