a safety case
Industries that handle serious hazards — nuclear plants, aircraft, medicines — are often required to produce a safety case: a written, structured argument, backed by evidence, that explains why a particular system is safe enough to operate, and which regulators and independent experts can scrutinize. The idea is now being adapted for AI: rather than pointing at a few passed tests, a developer assembles an explicit argument for why deploying a given model is acceptably safe.
A safety case is more than a pile of evaluation results. It states a claim ('this model does not pose an unacceptable risk of X'), lays out the reasoning that would make that claim true, and supplies the evidence for each step — capability and safety evaluations, red-teaming results, deployment safeguards, monitoring plans — while being honest about its assumptions and gaps. Crucially, it has to address why the evidence is sufficient: for instance, why passing these particular tests gives confidence about the untested situations the model will actually face.
The safety case is the discipline that takes the difference between 'passed the eval' and 'is safe' seriously. A score on a test is one piece of evidence; safety is a property of the whole deployed system in the real world, including adversaries, distribution shift, and capabilities we may have failed to elicit. A good safety case makes the leap from evidence to conclusion explicit and arguable, so weak links can be challenged. It is an emerging practice for frontier AI, not a mature or standardized one, and a safety case can still be wrong — but writing it forces the reasoning into the open where it can be checked.
Instead of saying 'the model passed our harm tests', a developer documents: here is the risk we are worried about, here is why our evaluations and safeguards together rule it out, here are the assumptions we are relying on — a structured argument an outside reviewer can attack.
A safety case argues from evidence to the conclusion 'safe enough', and exposes exactly where that argument could fail.
Passing evaluations is evidence inside a safety case, not the same as being safe: the case must argue why the tested cases justify confidence about the untested reality. Safety cases for AI are an emerging, not settled, practice and can still be mistaken.