Ko-WideSearch：面向网页代理穷举集合枚举的韩国广度搜索基准

摘要

Web-agent基准测试绝大多数衡量的是深度——即通过一系列约束条件定位一个晦涩难懂的答案——而广度，即穷举一个封闭集合并填充每个项目的属性，却很少被评估，尤其在英语之外的环境中。构建广度基准测试也很困难：验证一个黄金标准集是否完整且每个单元格正确，其成本远高于检查单个答案。我提出了Ko-WideSearch，一个通过自动化合成与验证流水线构建的韩语广度搜索基准测试。每个任务指定一个集合-父实体——例如一个电视剧季、一个朝代、一个联赛、一个行政区划、一次选举——并要求找出其全部成员以及每个项目的属性表，评分采用Item-F1、Column-F1和Row-F1。该基准测试涵盖190个实体、16个类别的228张表格，分为三个难度等级，由我独立调节的两个结构旋钮（表格宽度和二维复合键）设定，因此跨乘积的成员比例从0%逐渐上升到100%。在黄金标准集的构建和评分中使用同一个归一化感知比较器，从而避免仅因格式问题而过度丢弃稳定的日期和计数列。在二十个Web-agent上的测试结果显示，失败模式一致：智能体能够恢复集合，但无法恢复行（例如Item-F1为92.8，而Row-F1仅为53.7）；随着旋钮难度增加，准确率稳步下降，且增加搜索次数或投入更多资源均无法缩小差距。按单元格分析，困难在于找到正确的值而非格式化：开放式自由文本单元格失败率最高，而含标准答案（如日期或名称）的单元格通常表现正确。

English

Web-agent benchmarks overwhelmingly measure depth -- pinning one obscure answer behind a chain of constraints -- while breadth, exhaustively enumerating a closed set and filling each item's attributes, is barely evaluated, especially outside English. Breadth is also hard to build: certifying that a gold set is complete and every cell correct is far costlier than checking a single answer. I introduce Ko-WideSearch, a Korean breadth-search benchmark built by an automated synthesize-and-verify pipeline. Each task names a set-parent entity -- a TV season, a dynasty, a league, an administrative region, an election -- and asks for its full membership plus a per-item attribute table, graded by Item-, Column-, and Row-F1. It spans 228 tables over 190 entities and sixteen categories across three difficulty tiers, set by two structural knobs I dial independently -- table width and a 2-D composite key -- so cross-product membership climbs from 0\% to 100\% across the tiers. A single normalization-aware comparator is shared between gold construction and grading, so stable date and count columns are not over-dropped on formatting alone. Across twenty web agents, the failure is consistent: agents recover the set but not the rows (e.g.\ Item-F1 92.8 against Row-F1 53.7), accuracy falls steadily as the knobs harden, and neither more search nor more spend closes the gap. Broken down by cell, the hard part is finding the right value, not formatting it: open-ended free-text cells fail most, while cells with a standard answer such as a date or a name usually come out right.