What changes is clear: it is not currently possible to assign a defensible probability to machine-driven extinction. Prominent commentators offer estimates for p(doom), the chance artificial intelligence could destroy humanity, that span from 0 to more than 95 per cent. That spread means a single numeric probability has no scientific standing.
The Story in Pictures
That absence of a defensible p(doom) does not decide whether AI should be feared, only that plugging numbers into an equation cannot resolve the issue. The proper response is comparative: place the uncertainty about AI alongside risks where probability calculations rest on clear data or repeatable models, and treat each accordingly.
Some risks are straightforward to quantify. Buying a ticket in the UK National Lottery, for example, carries odds of about 1 in 4.9 of winning any prize, or 20.4 per cent, which implies a 79.6 per cent chance of losing. Those figures come from well-defined rules and many repeatable draws, so the odds are straightforward to calculate.
Personal safety risks are messier but still quantifiable at scale. The World Health Organization reports slightly more than 1 million people die in road traffic collisions each year. Dividing that by an 8 billion global population gives a crude personal odds estimate of roughly 0.01 per cent, though actual risk varies strongly by location, because, as the WHO notes, "more than 90% of road traffic deaths occur in low- and middle-income countries." Those numbers justify targeted precautions such as seatbelts and road safety policies.
Climate change offers another example of a complex yet modelled global risk. Using climate models to simulate alternative futures, researchers can make probabilistic statements about warming. The UN Environment Programme Emissions Gap Report concluded under current policies there is a 100 per cent chance of exceeding 1.5°C this century, a 92 per cent chance of exceeding 2°C and a 20 per cent chance of exceeding 3°C. Under its strongest assumptions about future action those chances fall to 79 per cent, 22 per cent and 0 per cent respectively, illustrating how policy choices change modeled probabilities.
At the other end of the spectrum, some catastrophic risks are easier to bound. NASA estimates we have found more than 90 per cent of asteroids above 1 kilometre in diameter and can predict their positions well enough to rule out any significant impact within the next century. Impacts from objects about 1 kilometre or larger are estimated to occur only once every 600,000 years, while the Chicxulub impactor that contributed to the dinosaurs’ extinction was roughly 10 kilometres wide. Smaller but still devastating objects, often called "city killers" at about 140 metres, are less well catalogued, but current estimates put their strike frequency at roughly once every 20,000 years.
Novel pandemic pathogens expose a different kind of uncertainty: a virus can appear and spread fast, as covid-19 showed, but that history does not automatically make its probability predictable. The core point is that AI doom currently sits closer to the unquantifiable end of this spectrum. Because p(doom) cannot presently be derived with scientific rigour, the best available approach is contextual: use established, modelled risks as reference points and prioritise measures that either reduce exposure to plausible harms or improve the empirical basis for future probability estimates.
That prescription shifts the task for governments, funders and researchers away from trying to defend a single p(doom) and toward two practical goals. First, focus on interventions that cut exposure to clearly plausible harms. Second, invest in empirical work that can turn currently speculative scenarios into modelled, testable risks. Those steps do not eliminate uncertainty, but they make policy choices more defensible and better aligned with where probability assessments are meaningful.
