p(doom)

In casual conversation, AI researchers sometimes ask each other a blunt question: 'what's your p(doom)?' It is shorthand for a personal probability estimate, how likely you think it is that advanced AI leads to catastrophe, often specifically human extinction or a permanent loss of human control. Read it as 'the probability of doom', a number between 0 and 1 (or a percentage) capturing one person's degree of belief.

The key thing to understand is what kind of number this is. p(doom) is not a measured frequency like the chance of rain, which we can check against records. There is no dataset of civilizations destroyed by AI to count. It is a subjective credence: an informed person's gut-plus-reasoning estimate about an unprecedented, one-time possibility. People also mean different things by it, over what time horizon, how bad an outcome counts as 'doom', whether they mean extinction or merely catastrophe, so two stated p(doom) numbers may not even be measuring the same event.

Reported values vary enormously, from tiny fractions of a percent to, for a few, more than half, and that spread is the honest headline, not any single figure. The number is useful mainly as a rough way to communicate how seriously someone takes the risk and to surface disagreement; it is not a scientific measurement and should never be cited as one. The responsible takeaway is the wide range and deep uncertainty among serious people, not a specific value to adopt or to mock.