Ko-WideSearch：一個用於網路智能體窮盡集合列舉的韓語廣度搜尋基準

摘要

網頁代理基準測試 overwhelmingly 側重於深度——將一個晦澀的答案埋藏在層層限制之後——而廣度，即窮舉一個封閉集合並逐一填寫每個項目的屬性，則鮮少被評估，尤其是在非英語環境中。此外，廣度的建構十分困難：驗證一個黃金集合的完整性並確保每個儲存格正確無誤，其成本遠高於檢查單一答案。為此，我提出了 Ko-WideSearch，一個基於韓語的廣度搜索基準測試，透過自動化的合成與驗證流程建構而成。每個任務指定一個集合父實體——如電視季、王朝、聯賽、行政區域、選舉——並要求列出其完整成員以及每個項目的屬性表，採用 Item-F1、Column-F1 及 Row-F1 進行評分。該基準測試涵蓋 228 個表格，分佈於 190 個實體與 16 個類別，並根據兩個獨立調整的結構旋鈕（表格寬度與二維複合鍵）劃分為三個難度層級，使得交叉乘積成員數從 0% 逐步上升至 100%。在黃金集合建構與評分過程中，使用一個統一的、具正規化感知的比較器，藉此避免因格式問題而過度剔除穩定的日期與計數欄位。在二十個網頁代理的測試中，結果呈現一致的失敗模式：代理能擷取出集合，卻無法正確提取行（例如 Item-F1 為 92.8，但 Row-F1 僅有 53.7），且隨著旋鈕難度提升，準確率持續下降，無論增加搜索次數或花費更多資源都無法縮小差距。進一步分析各儲存格後發現，困難點在於找出正確的值，而非格式問題：開放式自由文字儲存格的錯誤率最高，而日期或姓名等具有標準答案的儲存格則通常能正確處理。

English

Web-agent benchmarks overwhelmingly measure depth -- pinning one obscure answer behind a chain of constraints -- while breadth, exhaustively enumerating a closed set and filling each item's attributes, is barely evaluated, especially outside English. Breadth is also hard to build: certifying that a gold set is complete and every cell correct is far costlier than checking a single answer. I introduce Ko-WideSearch, a Korean breadth-search benchmark built by an automated synthesize-and-verify pipeline. Each task names a set-parent entity -- a TV season, a dynasty, a league, an administrative region, an election -- and asks for its full membership plus a per-item attribute table, graded by Item-, Column-, and Row-F1. It spans 228 tables over 190 entities and sixteen categories across three difficulty tiers, set by two structural knobs I dial independently -- table width and a 2-D composite key -- so cross-product membership climbs from 0\% to 100\% across the tiers. A single normalization-aware comparator is shared between gold construction and grading, so stable date and count columns are not over-dropped on formatting alone. Across twenty web agents, the failure is consistent: agents recover the set but not the rows (e.g.\ Item-F1 92.8 against Row-F1 53.7), accuracy falls steadily as the knobs harden, and neither more search nor more spend closes the gap. Broken down by cell, the hard part is finding the right value, not formatting it: open-ended free-text cells fail most, while cells with a standard answer such as a date or a name usually come out right.