AI models have likely reached parity with superforecasters on ForecastBench
New results from the ForecastBench leaderboard suggest the gap between frontier AI and elite human forecasters is closing.
In early July, Scott Alexander published a blog post claiming that “AI superforecasters are here.” In the post he explored the frontier of AI forecasting and discussed what it might look like to live in a world where bots reliably outperform top human forecasters. Underlying the whole post was a question: How do current AI systems compare to human superforecasters?
We run ForecastBench, a leading AI forecasting benchmark and the only one to compare AI forecasters to human superforecasters. This gives us a front-row seat to the race between AI and human forecasters. So, are the AI superforecasters here? Here’s what our results show.
Several models are already statistically indistinguishable from superforecaster-level accuracy
On the ForecastBench tournament leaderboard, several models are statistically indistinguishable from superforecaster-level accuracy.1 The top current system is from Cassi AI, with models from xAI and Google DeepMind also ranked as indistinguishable from superforecaster-level accuracy. The tournament leaderboard is intended to capture the frontier of LLM forecasting ability, so submissions may make use of tools, added context, fine-tuning, ensembling, or other methods.
We described the approach Cassi is taking in a previous blog post back in January. In essence, the approach uses a multi-stage pipeline that generates sub-questions and search queries, retrieves up-to-date context, and filters for relevance and recency. The system then generates forecasts using an ensemble of models, with another LLM analyzing the reasoning to produce a final forecast.
Although the overall architecture remains similar, Cassi has made some incremental improvements to the system since our January post. These include refinements to the forecasting pipeline, such as how the agent gathers, organizes, and synthesizes information, and better integration of relevant data sources.
Other systems have also likely reached parity with superforecasters on ForecastBench. These include external submissions from xAI and Google DeepMind.
An AI system has now ranked higher than superforecasters on market questions for the first time
On market questions, Cassi’s system also ranks higher than the superforecaster median forecast—the first time that any model has achieved this.2
For context, ForecastBench is divided into two different kinds of questions:
Dataset questions: Automatically generated from real-world time series (ACLED, DBnomics, FRED, Yahoo! Finance, and Wikipedia) using pre-specified templates.
Market questions: Drawn from leading prediction platforms (Manifold, Metaculus, Polymarket, and RAND Forecasting Initiative).
AI systems matched superforecasters on dataset questions back in May. However, in some ways market questions are a better test of human-like forecasting skill. LLMs may have an advantage over humans on dataset questions because they involve looking up lots of data, establishing base rates, and so on—tasks for which LLMs are uniquely well suited. Market questions often require judgment about novel, one-off events, so it’s significant that an AI system now outranks superforecasters on these questions.3
Many AI systems likely outperform superforecasters on dataset questions
17 submissions now rank above superforecasters on our preliminary leaderboard. The preliminary leaderboard ranks models based on their performance on resolved dataset questions. Given that until very recently Google DeepMind’s green-tree was the only submission ranked higher than superforecasters on this leaderboard, this is a significant development.
Our preliminary leaderboard provides early results before a model appears on the tournament leaderboard. While it takes 50 days after submission for a system to appear on the main tournament leaderboard, submissions appear on the preliminary leaderboard after at least 225 dataset questions have resolved (typically around 10 days after the initial submission). These rankings are not final, but our stability analysis suggests that these early results are robust. So, we expect many of these systems to rank higher than superforecasters on dataset questions once they appear on the tournament leaderboard.
Three submissions from Torchcast are at the top of the preliminary leaderboard. Torchcast told us their approach uses a proprietary multi-agent architecture to analyze each question, gather relevant evidence, generate forecasting perspectives, and produce a final probability estimate. They use a combination of Torchcast-trained models and commercially available LLMs.
Caveats
There are a few caveats that should be considered when interpreting all of these results:
We last elicited superforecaster predictions in 2024. Our comparison relies on a statistical extrapolation that grows less reliable over time.
Results depend on realized questions and are inherently stochastic. As more questions resolve, the results may shift.
The 95% CIs for many of these submissions overlap substantially; the results are more consistent with superforecaster parity than with outperformance.
These findings don’t mean that ForecastBench is saturated. The superforecaster median is a human reference point, and it is possible that AI systems may exceed human superforecaster accuracy.
Next steps
We’re working on a few ways to improve ForecastBench. Upcoming changes include:
A fresh superforecaster round (fall 2026)
Updated dataset questions (fall 2026)
Quantile questions (fall 2026)
For Cassi AI, the bootstrap one-sided p-value is 0.41, where the null hypothesis is that the model is equally accurate as superforecasters (i.e., we cannot reject equal forecasting accuracy at conventional significance levels). For xAI’s submissions the corresponding numbers are 0.16 and 0.15. For Google DeepMind’s submission the corresponding number is 0.14.
One anonymous submission (“Anomous 8, ensemble”) is also ranked higher than superforecasters on market questions. To view anonymous submissions, click the “show anonymous teams” toggle on the tournament leaderboard page.