Safety, Jailbreaks & Red-Teaming

LLM safety

An assistant that can write code, draft emails, and answer almost any question is also something a stranger can ask almost anything. LLM safety is the discipline of making sure that across that enormous space of possible requests, the model does not actively help with serious harm: building weapons, writing malware, planning violence, producing child sexual abuse material, or grooming and manipulating vulnerable people. The goal is not a model that refuses everything sensitive, but one that stays genuinely useful while declining the narrow set of requests where helping would cause real-world damage.

Safety is built in overlapping layers rather than a single switch. Pretraining data is filtered, the model is safety fine-tuned to refuse clearly harmful requests, a system prompt and usage policy frame what it should and should not do, separate input and output filters catch what slips through, and human red-teaming plus automated evaluation continually probe for failures. No layer is perfect, so they are stacked precisely so that an attack must defeat several at once.

It helps to be honest about scope. Safety is not the same as alignment, capability, or fairness, though they overlap; it is specifically about preventing harm. And it is never finished: every new capability, language, and use case opens new failure modes, so safety is treated as ongoing measurement of residual risk, not a property you certify once.

Safety constrains behavior, not knowledge — a model can be made to refuse a topic it still understands in full detail, which is why determined attackers keep probing.

Also called
model safetyAI safety for language models