PluraMath：将数学推理评估拓展至高资源语言之外

摘要

数学推理已成为评估和调优推理大语言模型（LLMs）的核心任务，但现有基准测试仍严重偏向高资源语言，英语和中文在预训练语料库和评估套件中占据主导地位。最近发布的PolyMath（Wang等人，2025）数据集虽迈出了重要一步，但其覆盖范围仍仅限于18种高资源语言。为弥补这一不足，我们提出了PluraMath，将PolyMath扩展至另外18种代表性不足的语言，涵盖6个语系——从中等资源到极低资源场景。我们通过人工筛选流程构建数据集，由母语者仔细验证预计算翻译结果。利用PluraMath，我们随后对跨越四个模型规模（小型、中型、大型和闭源集成系统）的27个推理大语言模型进行了基准测试，探究了最先进模型在不同语言条件下的多语言数学推理能力。我们的细粒度分析证实，高资源语言与代表性不足语言在数学推理性能上仍存在持续差距，而更强的结果通常与更好的指令遵循能力相关。我们完全开源了数据集、数据采集流程和评估框架，旨在降低代表性不足社区开发多语言基准的门槛。

English

Mathematical reasoning has become a central task for evaluating and tuning reasoning Large Language Models (LLMs), yet existing benchmarks remain heavily biased toward high-resource languages, with English and Chinese dominating both pre-training corpora and evaluation suites. The recently released PolyMath (Wang et al., 2025) dataset represents a significant step forward, yet its coverage is still limited to 18 only high-resource languages. To address this gap, we introduce PluraMath, an extension of PolyMath to 18 additional {underrepresented languages spanning 6 language families -- ranging from mid-resource to extreme low-resource settings. We constructed the dataset through a human-curated pipeline, where native speakers thoroughly validated pre-computed translations. Using PluraMath, we then benchmark 27 reasoning LLMs across four model scales -- small, mid-size, large, and closed-source ensembles -- probing the multilingual mathematical reasoning capabilities of state-of-the-art models under diverse linguistic conditions. Our fine-grained analysis confirms a persistent gap in mathematical reasoning performance between high-resource and underrepresented languages, with stronger results largely associated with better instruction-following ability. We fully open-source our dataset, data acquisition pipeline, and evaluation framework, with the goal of lowering the barrier to multilingual benchmark development for underrepresented communities.