WorldCupArena：針對語言模型及深度研究代理在足球預測上的細粒度評估

摘要

预测一场足球比赛的结果在开球前需要的不仅仅是了解历史战绩：模型必须利用动态变化的信息，并在答案尚未揭晓时给出明确的预测。我们提出WorldCupArena，这是一个面向语言模型和深度研究智能体的动态基准测试。2026年国际足联世界杯是它的首次评估，同样的流程可复用于未来的联赛和杯赛。每场比赛前，模型要么接收一份公共证据包，要么自行搜索信息。它预测比赛结果与比分、可能的球员与事件、比赛统计数据以及赛事的最终走向。比赛结束后，这些预测将与实际记录的结果进行对比。我们报告比赛结果准确率、精确比分准确率，以及一种比分线评分（当预测比分接近但不完全精确时给予部分分数），同时还包括其他预测任务的评分。在包含104场比赛和13个系统的测试中，结果准确率相似的模型在细节预测上表现出更明显的差异。与博彩市场和人类球迷基线相比，最佳系统在比赛结果和精确比分准确率上只有小幅提升，但在比分线评分上提升更为明显。新的赛程可在开始时添加，使得该基准测试能够在无需使用已知结果的情况下评估未来模型。代码、提示词、预测结果以及评估脚本已在 https://github.com/wzk1015/WorldCupArena 开源。

English

Predicting a football match before kickoff requires more than knowing past results: a model must use changing information and make a clear prediction before the answer is available. We present WorldCupArena, a dynamic benchmark for language models and deep-research agents. The 2026 FIFA World Cup is its first evaluation, and the same process can be reused for future leagues and cups. Before each match, a model either receives a common evidence package or searches for information itself. It predicts the result and score, likely players and events, match statistics, and the outcome of the competition. After the match, these predictions are compared with the recorded result. We report result accuracy, exact-score accuracy, and a scoreline score that gives some credit when a predicted score is close but not exact, together with scores for the other prediction tasks. Across 104 matches and 13 systems, models with similar result accuracy differ more clearly on detailed predictions. Compared with betting-market and human-fan baselines, the best system shows only small gains in result and exact-score accuracy, but a clearer gain in Scoreline. New schedules can be added as they begin, allowing the benchmark to evaluate future models without using outcomes that are already known. Code, prompts, predictions, and evaluation scripts are open sourced at https://github.com/wzk1015/WorldCupArena.