DoTA-RAG：動態思維聚合檢索增強生成

摘要

本文介紹了DoTA-RAG（動態思維聚合檢索增強生成系統），這是一個針對高吞吐量、大規模網絡知識索引優化的檢索增強生成系統。傳統的RAG管道在處理海量、多樣化數據集時，常面臨高延遲和精度受限的問題。DoTA-RAG通過三階段管道應對這些挑戰：查詢重寫、動態路由至專用子索引，以及多階段檢索與排序。我們進一步通過評估並選擇更優的嵌入模型，對大型FineWeb-10BT語料庫進行重新嵌入，從而增強了檢索效果。此外，我們利用DataMorgana設置生成了一個包含500個問題的多樣化問答數據集，涵蓋了WebOrganizer的廣泛主題和格式。DoTA-RAG將答案正確率從0.752（基線，使用LiveRAG預構建的向量存儲）提升至1.478，同時保持低延遲，並在Live Challenge Day上取得了0.929的正確率。這些結果凸顯了DoTA-RAG在需要快速、可靠訪問大規模且不斷演進的知識源領域中的實際部署潛力。

English

In this paper, we introduce DoTA-RAG (Dynamic-of-Thought Aggregation RAG), a retrieval-augmented generation system optimized for high-throughput, large-scale web knowledge indexes. Traditional RAG pipelines often suffer from high latency and limited accuracy over massive, diverse datasets. DoTA-RAG addresses these challenges with a three-stage pipeline: query rewriting, dynamic routing to specialized sub-indexes, and multi-stage retrieval and ranking. We further enhance retrieval by evaluating and selecting a superior embedding model, re-embedding the large FineWeb-10BT corpus. Moreover, we create a diverse Q&A dataset of 500 questions generated via the DataMorgana setup across a broad range of WebOrganizer topics and formats. DoTA-RAG improves the answer correctness score from 0.752 (baseline, using LiveRAG pre-built vector store) to 1.478 while maintaining low latency, and it achieves a 0.929 correctness score on the Live Challenge Day. These results highlight DoTA-RAG's potential for practical deployment in domains requiring fast, reliable access to large and evolving knowledge sources.

DoTA-RAG：動態思維聚合檢索增強生成

DoTA-RAG: Dynamic of Thought Aggregation RAG

摘要

Support