评估大语言模型在现实世界预测中与人类超级预测者的对比表现
Evaluating LLMs on Real-World Forecasting Against Human Superforecasters
July 6, 2025
作者: Janna Lu
cs.AI
摘要
大型语言模型（LLMs）在多种任务中展现了卓越的能力，但其预测未来事件的功能仍待深入探究。一年前，大型语言模型在准确性上还难以匹及人类群体的预测水平。我基于Metaculus平台上的464个预测问题，对当前最先进的LLMs进行了评估，并将其表现与人类超级预测者进行了对比。前沿模型虽在Brier分数上看似超越了普通人群，但与超级预测者群体相比，仍存在显著差距。
English
Large language models (LLMs) have demonstrated remarkable capabilities across
diverse tasks, but their ability to forecast future events remains
understudied. A year ago, large language models struggle to come close to the
accuracy of a human crowd. I evaluate state-of-the-art LLMs on 464 forecasting
questions from Metaculus, comparing their performance against human
superforecasters. Frontier models achieve Brier scores that ostensibly surpass
the human crowd but still significantly underperform a group of
superforecasters.