StatEval：大型語言模型在統計學中的全面基準測試

摘要

大型语言模型（LLMs）在数学与逻辑推理方面展现了显著的进步，然而，作为一门独特且综合的学科，统计学在基准测试中的探索仍显不足。为填补这一空白，我们推出了StatEval，这是首个专为统计学设计的全面基准测试，覆盖了从基础到高级的广泛难度层次。StatEval包含13,817道涵盖本科及研究生课程的基础题目，以及从顶尖期刊中提取的2,374项研究级证明任务。为构建此基准，我们设计了一个可扩展的多代理流程，结合人类参与验证，自动化实现大规模题目提取、重写及质量控制，同时确保学术严谨性。此外，我们提出了一套针对计算与证明任务量身定制的稳健评估框架，支持对推理能力进行细致评估。实验结果显示，尽管闭源模型如GPT5-mini在研究级问题上得分低于57%，开源模型的表现则更为逊色。这些发现凸显了统计推理的独特挑战及当前LLMs的局限性。我们期待StatEval能成为推动大型语言模型统计智能发展的严格基准。所有数据与代码均可在我们的网络平台上获取：https://stateval.github.io/。

English

Large language models (LLMs) have demonstrated remarkable advances in mathematical and logical reasoning, yet statistics, as a distinct and integrative discipline, remains underexplored in benchmarking efforts. To address this gap, we introduce StatEval, the first comprehensive benchmark dedicated to statistics, spanning both breadth and depth across difficulty levels. StatEval consists of 13,817 foundational problems covering undergraduate and graduate curricula, together with 2374 research-level proof tasks extracted from leading journals. To construct the benchmark, we design a scalable multi-agent pipeline with human-in-the-loop validation that automates large-scale problem extraction, rewriting, and quality control, while ensuring academic rigor. We further propose a robust evaluation framework tailored to both computational and proof-based tasks, enabling fine-grained assessment of reasoning ability. Experimental results reveal that while closed-source models such as GPT5-mini achieve below 57\% on research-level problems, with open-source models performing significantly lower. These findings highlight the unique challenges of statistical reasoning and the limitations of current LLMs. We expect StatEval to serve as a rigorous benchmark for advancing statistical intelligence in large language models. All data and code are available on our web platform: https://stateval.github.io/.

StatEval：大型語言模型在統計學中的全面基準測試

StatEval: A Comprehensive Benchmark for Large Language Models in Statistics

摘要

Support