搜尋代理何時應提問：用於具澄清意識深度搜索的 DiscoBench

摘要

大型語言模型驅動的搜尋代理正被廣泛應用於處理複雜的資訊搜尋任務，這類任務需要透過多步驟檢索與推理來達成使用者目標。然而，現有基準測試通常假設使用者的查詢是完整且明確的，忽略了現實世界中搜尋請求往往模糊、未充分指定，甚至包含事實錯誤。在深度搜尋情境下，此類模糊性可能沿著多步推理鏈傳播，導致代理偏離正確的搜尋軌跡。為彌補此一缺口，我們提出DiscoBench，一個專為具澄清意識的深度搜尋所設計的基準測試，旨在評估搜尋代理能否主動識別模糊性、提出有效的澄清問題，並透過與使用者互動來恢復正確的推理路徑。DiscoBench涵蓋211個樣本及463個模糊性實例，橫跨11個真實世界領域，並包含四種模糊類型。我們進一步設計了一個適用於多輪互動的使用者模擬器，並從四個面向評估模型表現：任務實用性、模糊性檢測、互動策略及成本效率。在代表性大型語言模型上的實驗顯示，模糊性檢測與有效澄清是兩種截然不同的能力，而反覆搜尋而非請求澄清的表現往往不如直接猜測，這凸顯了當前搜尋代理在檢索能力與互動式問題解決之間存在的關鍵落差。

English

Search agents powered by large language models (LLMs) are increasingly used to solve complex information-seeking tasks, requiring multi-step retrieval and reasoning to fulfill user goals. However, existing benchmarks often assume that user queries are complete and explicit, overlooking the fact that real-world search requests are frequently vague, underspecified, or even factually incorrect. In deep search scenarios, such ambiguity can propagate along multi-step reasoning chains and lead agents toward incorrect search trajectories. To address this gap, we introduce DiscoBench, a benchmark for clarification-aware deep search, designed to evaluate whether search agents can proactively identify ambiguity, ask effective clarification questions, and recover correct reasoning paths through user interaction. DiscoBench contains 211 samples and 463 ambiguity instances across 11 real-world domains, covering four ambiguity types. We further design a user simulator for multi-turn interaction and evaluate model performance from four perspectives: task utility, ambiguity detection, interaction strategy, and cost efficiency. Experiments on representative LLMs show that ambiguity detection and effective clarification are distinct capabilities, and that repeatedly searching instead of asking for clarification often performs worse than direct guessing, highlighting a critical gap between retrieval ability and interactive problem-solving in current search agents.