檢索更佳，穩健性卻更差：多跳RAG如何放大上游ASR錯誤

摘要

基於語音的應用程式在進入任何檢索模組之前，會先透過自動語音辨識（ASR）處理口語查詢，因此 ASR 錯誤會作為固定的上游限制進入流程。我們以實證方式測試標準檢索增強生成（RAG）的兩種擴充——實體圖譜連結與迭代式重述——是吸收還是放大這些錯誤。使用透過神經 TTS 合成的四種英語口音，我們在三個多跳問答基準（HotpotQA、2WikiMultiHopQA 和 MuSiQue）上評估四種 RAG 配置，並與乾淨文本的 oracle 對照進行比較。儘管結構上更豐富的配置在 ASR 輸入下通常保留較高的絕對 F1，但兩種擴充都會放大錯誤：在所有三個基準上，其組合在乾淨文本與最高 WER 口音之間的 F1 差距比單純的密集檢索高出 36–67%。主要的失敗模式是一個或多個查詢實體被破壞，這在 2WikiMultiHopQA 上佔所有四種方法退化案例的 87–96%。兩種輕量級的表面形式緩解措施幾乎未縮小這一差距，表明下游檢索結構會放大剩餘的實體錯誤。我們在 https://github.com/ZhenghuaBao/spoken-multihop-rag 發布程式碼與資料。

English

Speech-based applications pass spoken queries through automatic speech recognition (ASR) before any retrieval module, so ASR errors enter the pipeline as a fixed upstream constraint. We empirically test whether two extensions to standard retrieval-augmented generation (RAG), entity-graph linking and iterative reformulation, absorb or amplify these errors. Using four English accents synthesized through neural TTS, we evaluate four RAG configurations on three multi-hop QA benchmarks (HotpotQA, 2WikiMultiHopQA and MuSiQue) against a clean-text oracle. Although the structurally richer configurations generally retain higher absolute F1 under ASR input, both extensions amplify the error: the F1 gap from clean text to the highest-WER accent is 36-67% larger under their combination than under naive dense retrieval, on all three benchmarks. The dominant failure mode is corruption of one or more query entities, accounting for 87-96% of degradation cases on 2WikiMultiHopQA across all four methods. Two lightweight surface-form mitigations leave most of the gap intact, indicating that downstream retrieval structure amplifies remaining entity errors. We release code and data at https://github.com/ZhenghuaBao/spoken-multihop-rag .