structural risk

Some harms cannot be pinned on any single broken part or any single villain. Traffic congestion, financial crashes, and arms races all emerge from many actors each behaving sensibly given their incentives, yet producing a bad collective outcome. Structural risk in AI is harm of this shape: it comes from how AI changes the incentives, power balances, and competitive dynamics of society, even when every individual system works as intended and no one is acting maliciously. It is the third standard category, beyond accident (unintended failure) and misuse (intentional harm).

Here the danger lives in the system of actors, not in one model. Consider racing dynamics: if several labs or nations fear being second, each has an incentive to deploy faster and cut safety corners, so competition itself raises everyone's risk, with no single party to blame. Other examples include gradual concentration of economic or political power in whoever controls the most capable AI, erosion of a shared sense of what is true as synthetic media floods the information space, or critical institutions quietly becoming dependent on systems no one fully understands. Each individual step can look locally reasonable while the trajectory is collectively harmful.

Structural risk is the hardest of the three to address because there is no single device to fix or bad actor to stop; the fix has to be partly social and institutional. This is the home turf of AI governance, compute governance, and ideas like international coordination and differential technological development, attempts to change the incentives so that safer choices are also competitively viable. It is also the category most easily overlooked by purely technical safety work, which is why many argue technical and governance approaches have to advance together.