評估大型語言模型在現實世界預測中對抗人類超級預測者的表現
Evaluating LLMs on Real-World Forecasting Against Human Superforecasters
July 6, 2025
作者: Janna Lu
cs.AI
摘要
大型語言模型（LLMs）在多樣化的任務中展現了卓越的能力，但其預測未來事件的能力仍未被充分研究。一年前，大型語言模型在準確性上難以接近人類群體的預測水平。我評估了最先進的LLMs在Metaculus上的464個預測問題，並將其表現與人類超級預測者進行比較。前沿模型所獲得的Brier分數表面上超越了人類群體，但仍顯著落後於一組超級預測者。
English
Large language models (LLMs) have demonstrated remarkable capabilities across
diverse tasks, but their ability to forecast future events remains
understudied. A year ago, large language models struggle to come close to the
accuracy of a human crowd. I evaluate state-of-the-art LLMs on 464 forecasting
questions from Metaculus, comparing their performance against human
superforecasters. Frontier models achieve Brier scores that ostensibly surpass
the human crowd but still significantly underperform a group of
superforecasters.