Summary
Over the last few months, AI risk has burst into the public and political consciousness in a big way. A combination of real cyber incidents, high-profile resignations, and statements from lab leaders has brought the discussion of catastrophic risk to a level of prominence that just a year ago seemed highly improbable.
While there are plenty of catastrophic risk estimates out there (including from our forecasters), we know much less about what people think about how policy can address risk from AI. Is AI risk addressable by policy? If so, which are the right policy levers to pull, and what is the likelihood of them being implemented?
The Longitudinal Expert AI Panel (LEAP) brings together forecasts from the kinds of experts that policymakers look to when considering the future of AI: computer scientists, AI industry experts, top economists, policy experts, and AI risk experts. In Wave 12 we asked our panelists to consider how far various policies would go toward addressing catastrophic risks from AI. We also asked panelists which policies they supported, and which ones they thought had the highest chances of being implemented.
We’ve also recruited a new expert sample, composed of people who work on addressing the risks posed by transformative AI. In this and subsequent work, we’ll report results from this “AI risk experts” group as well as our other experts, superforecasters, and our representative sample of the US general public.
The results from our policy forecasts are encouraging. LEAP panelists believe that specific policies can substantially reduce the risk of AI-caused catastrophes, and experts largely agree on which policies should be implemented, despite their disparate professional backgrounds and views on AI. Across groups, clear majorities support establishing an international body with pre-release authorization power over frontier models and expect doing so to have among the largest risk-reducing effects. Majorities of most groups also support policies to slow AI progress and strengthen AI labs’ liability for harms caused by their models, policies most forecasters also expect to meaningfully reduce risk.
However, when it came to implementing these policies, forecasters were much more pessimistic. The policies most expected to reduce risk were also the ones that respondents thought were least likely to be implemented. The one policy that was associated with an increase in risk—federal preemption of state AI laws—was seen as the most likely to pass. Panelists also noted throughout that although bilateral (US–China) versions of policies were more likely to reduce catastrophic risk from AI, they were also less likely to be implemented.
In essence, this wave highlights a contradiction at the heart of the AI policy debate. Forecasters believe that policy really can reduce risks from AI, but are pessimistic that governments will put in place the policies that have the highest impact.
Read on for more detailed insights from our wave of policy forecasts. You can also read the full report, with a question-by-question breakdown and forecaster rationales on the LEAP website.
Policies
We asked forecasters to consider the following policies:1
P1: Federal preemption of state AI laws
A US federal statute or action barring states from regulating AI more strictly than the federal government.
P2: AI capability slowdown
Unilateral decision by US policymakers or a bilateral/multilateral decision made jointly by policymakers in China and the United States to slow the development of frontier AI systems.
P2.a: US enacts a policy to slow AI development, with no qualifying measure binding the US and China in force.
P2.b: US and China agree to a single bilateral or multilateral measure to slow AI development.
P3: Pre-release authorization for frontier models
A binding requirement that frontier models be vetted and authorized before release, imposed by a US federal body or by an international body.
P3.a: A US federal body holds pre-release authorization power.
P3.b: An international body whose member jurisdictions include the US and China holds pre-release authorization power over frontier models.
P4: Strict liability
A US federal statute treating frontier model training and deployment as an abnormally dangerous activity subject to strict liability (i.e., a regime under which a developer is liable for harms its models cause without the injured party having to prove the developer was careless or the model defective).
P5: Bundle of policies (P2.b, P3.b, and P4)
Assume all three policies are in force: a binding US–China measure slowing frontier AI development, an international body (with member jurisdictions including the US and China) holding pre-release authorization power over frontier models, and federal strict liability for frontier AI harms.
Key insights
Respondents disagree significantly about how risky AI is, yet still largely agree on what to do about it
We started by asking forecasters to give us their baseline all-things-considered forecasts of the likelihood of an AI-caused catastrophe that leads to the deaths of more than 10% of humans by 2050. Here’s what they forecast:2
Experts: 2%
Superforecasters: 1.6%
AI risk experts: 5.2%
General public: 5%
As these forecasts show, there is wide disagreement about the baseline level of risk. AI risk experts give risk estimates that are more than double those of our general expert sample, and three times higher than superforecasters.
Despite this disagreement about the level of baseline risk, panelists largely agreed about which AI risk policies should be implemented. A majority of forecasters in each group agreed that the following four policies should be implemented:3
Pre-release authorization requirement (international body, including US and China)
Pre-release authorization requirement (US-only)
Strict liability for developers
AI capability slowdown (US–China)
The only policy that did not win majority support in any group of forecasters was a federal preemption of state AI laws. This was supported by just 29% of experts, 14% of superforecasters, 20% of AI risk experts, and 50% of the general public. The most divisive policy was a US-only capability slowdown—34% of experts and 35% of superforecasters thought this policy should be implemented, compared with 60% of AI risk experts and 65% of the general public.
Respondents across groups believe policy can substantially reduce catastrophic AI risk
After they gave us their baseline forecasts of AI-caused catastrophic risk, we asked forecasters to predict the same outcome but this time to assume that each of the policies described above was implemented immediately by the relevant governing body.
Experts forecast lower catastrophic risk under five of seven policies in our policy menu, indicating strong belief across forecasters that policies can address a substantial portion of AI-caused catastrophic risk.4 Experts and superforecasters expected an international pre-release authorization body for frontier models to be the most effective single policy at reducing risk, while AI risk experts predicted the largest effects from a US–China slowdown agreement.
All groups expected large reductions in risk from an international pre-release regime or a US–China slowdown agreement, forecasting that both policies would reduce risk by between 20% and 50% compared with the baseline. The three-policy bundle of US–China capability slowdown, international pre-release authorization, and strict liability was associated with the largest decrease in risk of any policy we tested—a forecast 72% reduction in risk from experts, 62% reduction from AI risk experts and a 56% reduction from superforecasters.
Forecasters strongly support the implementation of an international body with pre-release authorization power over frontier models
All groups strongly supported a policy that would give an international body whose member jurisdictions include the US and China pre-release authorization power over frontier models. In terms of support for implementation, this was the most-favored policy by experts (75% in favor), superforecasters (86% in favor), and the general public (84% in favor). For AI risk experts this was the second-most-favored policy (86% in favor), narrowly behind a US–China capability slowdown, which 87% of that group favored.
Forecasters also agreed that an international pre-release authorization regime would significantly reduce the risk of an AI-caused catastrophe by 2050. Compared with their baseline risk probabilities, forecasts of an AI-caused catastrophe fell by at least 23% when conditioned on implementation of this policy:
Experts: 2% → 1% (-50%)
Superforecasters: 1.6% → 1% (-38%)
AI risk experts: 5.2% → 4% (-23%)
General public: 5% → 3% (-40%)
For experts, superforecasters, and the general public, only the policy bundle was associated with a greater decrease in AI-caused catastrophic risk.
US-only policies are forecast to significantly reduce AI risk, but are still seen as less effective than bilateral policy
For two policies, we asked forecasters to predict the effect of a version implemented by the US only, and a version implemented bilaterally or multilaterally by the US and China. These were a decision by policymakers to slow the development of frontier AI systems (P2), and a binding requirement that frontier models be vetted and authorized before release, imposed by a US federal body or by an international body (P3).
The bilateral version of both of these policies was associated with a larger reduction in catastrophic risk compared with a US-only version of the policy. For example, experts gave a baseline 2% chance of an AI-caused catastrophe by 2050. This risk was unchanged if a US-only AI capability slowdown was implemented, and fell to 1.2% if this slowdown was implemented in both the US and China. For the pre-release authorization policy, the chance of an AI catastrophe fell to 1.5% (US-only version) and 1% (international version). This same pattern holds true across forecaster groups and when forecasters assess the chance of a smaller “sub-catastrophe,” which causes at least 1,000,000 deaths within a five-year period.
Despite bilateral policies being viewed as the most impactful, forecasters thought they were less likely to be implemented when compared with US-only versions of the same policy. Experts assigned a 20% probability to a pre-release authorization policy being implemented in the US before the end of 2030, compared with a 5% chance of an international version of the policy being implemented by the same date. For the AI capability slowdown policy, experts gave an 8% chance of a US-only version being implemented and a 5% chance to a bilateral version being implemented.
Experts strongly oppose federal preemption and think its passage would increase AI risk. But they also think it's the policy most likely to pass
A federal preemption of state AI laws was the only policy associated with an increase the risk of an AI-caused catastrophe by 2050 by all forecaster groups. These are the median forecasts of each group in the unconditional scenario, and then conditional on a federal preemption of state AI laws being passed, with the increase in risk in such a scenario shown in brackets:
Experts: 2% → 3% (+50%)
Superforecasters: 1.6% → 2.5% (+56%)
AI risk experts: 5.2% → 8% (+54%)
General public: 5% → 6.9% (+38%)
Federal preemption also had the lowest support for implementation of any of the policies we tested. Just 29% of experts supported implementation, with 14% of superforecasters, 20% of AI risk experts, and 50% of the general public in support. Despite this, experts assigned this policy the highest likelihood of passing—forecasting a 26% chance that such a policy is implemented by the end of 2030, rising to 40% by the end of 2050. This policy was also forecast as the most likely to pass by 2030 by the general public, with superforecasters and AI risk experts placing slightly more likelihood on pre-release authorization (US only) being implemented by 2030.
In their rationales, forecasters noted that companies often choose to comply with the strictest state standard, setting a national "floor" for safety standards, which a federal preemption would remove. Several forecasters argued that preemption's impact on catastrophic risk would be minimal, as much of the state-level regulation that might be preempted was unlikely to focus on catastrophic risk, and instead would address other concerns such as discrimination, bias, data exploitation, and so on.
The policies expected to reduce risk the most are those respondents think least likely to be implemented
We asked forecasters to predict the probability that the relevant governing body/bodies will implement each policy by the end of 2030 and 2050. Across all groups, most respondents place the probability of implementation by the end of 2030 at or below 25% for almost all policies.
Experts predict the largest reductions in risk under international pre-release approval, US–China slowdown, and strict liability. They also think these policies are least likely to be implemented, giving all three a 5% chance of being in place by 2030.
Conversely, experts give the highest chance of implementation to federal preemption (26%), under which they expect risk to increase.
Despite placing a high probability of implementation of federal preemption and US pre-release authorization, forecasters expect these policies to be among the least effective at reducing catastrophic AI risk. In contrast, the bundle of three policies, which is ranked as the most effective at reducing catastrophic AI risk compared to the baseline by all groups, is seen as the least likely to be implemented. For instance, experts expect the bundle of policies to reduce catastrophic AI risk by nearly 72%, but give only a 1% chance of these policies being implemented. This trend is seen across other policies where forecasters predict catastrophic AI risk will be lowered, including capability slowdowns (P2), pre-release authorization, and strict liability (P4).
Comparing EA-affiliated forecasters with other forecasters
Recent discourse about AI risk has focused on effective altruism (EA) and the differences in beliefs between EA-influenced individuals and other groups. We anticipated this discussion when designing LEAP, and we can explore this dynamic in our LEAP sample by splitting respondents into two groups:
1) Experts with connections to EA, including grant recipients and employees of EA-adjacent organizations, based on our sample definition last year5
2) Experts with no connection to EA
The figure below shows that these two groups hold virtually identical preferences about which policies they think should be implemented and have directionally similar beliefs about the impact of policies on catastrophic risk.
In other words, our headline results are not driven by experts connected to effective altruism. Instead, they represent a broadly shared view of a representative sample of experts.
Find out more
For full results from Wave 12, visit the LEAP website.
This list covers several options that are currently prominent in the discourse, but is certainly non-exhaustive. Among other things, it does not cover policies aimed at improving societal resilience or response to catastrophe. Future LEAP waves will likely address other types of policy that could not fit into this wave’s survey.
Note that “baseline” refers to forecasters' median unconditional forecasts (their all-things-considered predictions that include what policies they expect to be implemented). We also had forecasters predict risk under a strong “status quo” assumption of no meaningful policy change. These estimates are uniformly higher, implying that all groups expect some effective policy to be implemented.
It is possible that forecasters were primed to be more likely to support these policies by their inclusion in a survey focused on risk. We did ask forecasters to consider and name tradeoffs that apply to these policies, but the balance of our questions focused on risk, making a priming effect plausible.
In this report we compare median forecasts under different conditions as our main analysis of the effects of those conditions. For example, we compare the median risk prediction across experts under international pre-release restrictions to the median risk prediction across experts under the unconditional scenario to estimate the effect of international pre-release restrictions according to experts. We could also use the median individual’s difference in risk between the two conditions to answer the same question. The specific numbers sometimes change under this alternative method, but not the overall takeaways presented here.
See the LEAP launch paper for the full definition of our EA-affiliated group










Copying a question someone asked on X that I'm also curious about: "63% ea affiliated experts support pre release authorization but their median forecast is that this would lead to +20% catastrophic risk? what?"
If I would do just one thing it would be around Liability. You break it you own it. You hack Hugging 🤗 Face etc you are liable. Clearly civilly and maybe criminally