阿尔法卓越基准

Alpha Excel Benchmark

May 7, 2025

作者: David Noever, Forrest McKee

cs.AI

摘要

本研究提出了一种新颖的基准测试方法，利用源自“金融建模世界杯”（FMWC）Excel竞赛的挑战来评估大型语言模型（LLMs）。我们介绍了一种将113项现有FMWC挑战转化为可编程评估的JSON格式的方法，并运用此数据集对比了多个领先LLMs的表现。研究结果显示，不同挑战类别下模型性能存在显著差异，模型在模式识别任务上展现出特定优势，但在复杂数值推理方面则面临挑战。该基准测试为评估LLMs在现实商业导向任务中的能力提供了一个标准化框架，而非局限于抽象学术问题。通过确立微软Excel日常用户——全球15亿人——的熟练度作为连接学术AI基准与实际商业应用的有意义评价指标，本研究为不断发展的AI基准测试领域做出了贡献。

English

This study presents a novel benchmark for evaluating Large Language Models (LLMs) using challenges derived from the Financial Modeling World Cup (FMWC) Excel competitions. We introduce a methodology for converting 113 existing FMWC challenges into programmatically evaluable JSON formats and use this dataset to compare the performance of several leading LLMs. Our findings demonstrate significant variations in performance across different challenge categories, with models showing specific strengths in pattern recognition tasks but struggling with complex numerical reasoning. The benchmark provides a standardized framework for assessing LLM capabilities in realistic business-oriented tasks rather than abstract academic problems. This research contributes to the growing field of AI benchmarking by establishing proficiency among the 1.5 billion people who daily use Microsoft Excel as a meaningful evaluation metric that bridges the gap between academic AI benchmarks and practical business applications.