JOVANA
Explore Library Glossary Getting Started Three Levels Fields How it works Mission
Join the mission
All guides

What "LLM safety" actually means

Safety is not one switch but a stack of overlapping defenses. Meet the layers and the central tension that shapes all of them.

A helpful model is a dangerous default

A base language model is trained to continue text, and a chat model is then tuned to be helpful — to do what you ask. Pure helpfulness is exactly the problem: a model that will cheerfully draft anything will also draft a phishing email, a working exploit, or step-by-step instructions for something genuinely dangerous. LLM safety is the whole engineering and policy effort to keep a model useful while making it decline the small slice of requests that cause real-world harm — without turning it into a useless scold for everyone else.

Defense in depth: the safety stack

No single mechanism is reliable on its own, so safety is built in layers, each catching what the others miss. The deepest layer is baked into the weights; the outer layers wrap the model at runtime.

  1. Safety fine-tuning — teaching the model itself to refuse, via demonstrations of good refusals plus preference training (RLHF, Constitutional AI). This shapes the model's own judgment.
  2. The system prompt — instructions from the developer (system-prompt design) that set rules and tone before the user ever speaks.
  3. Guardrails and filters — separate classifiers that screen the input and the output, independent of the main model.
  4. Content moderation — category-level classification of harmful content (violence, CSAM, self-harm, etc.) at the platform boundary, plus human review and incident response.
The deepest layer is safety fine-tuning via RLHF: human preferences train a reward model that tunes the policy toward safe behavior.

Diagram of RLHF: human preference comparisons feed a reward model that guides policy tuning.

Because each layer is imperfect, an attacker has to defeat all of them at once. That is the whole point: redundancy buys you safety margin that no single layer can.

Refusal is a learned behavior, not a rule

It is tempting to imagine a hard-coded list of forbidden topics. There is no such list inside the weights. Refusal is a statistical behavior the model learned: given a request that resembles its training examples of harmful asks, it becomes likely to produce a polite decline instead of compliance. That has two consequences. First, refusal can be steered — phrase a request so it no longer resembles the training distribution and the learned trigger may not fire. Second, refusal can over-fire: the model declines benign requests that merely pattern-match to dangerous ones (asking about the chemistry of a kitchen smell, say). Both failure modes matter, and they pull in opposite directions.

Refusal is not a hard-coded rule but a next-token behavior: in the generation loop a risky-looking request just makes refusal-style tokens more probable.

Autoregressive generation loop feeding each predicted token back as input for the next step.

The tension that never goes away

Every decision in this track sits on one axis: the safety-versus-helpfulness trade-off. Tighten the model and false refusals climb, frustrating doctors, security researchers, and novelists with legitimate needs. Loosen it and genuinely harmful completions slip through. You cannot maximize both; you choose a point on a frontier and try to push the whole frontier outward with better technique. Mature safety work is honest about this — it reports both the harm rate and the over-refusal rate, never just the flattering one.

\max_{\theta}\; \mathbb{E}\!\left[\text{helpfulness}\right] \;-\; \lambda\,\mathbb{E}\!\left[\text{harm}\right]

The whole track sits on one knob lambda: raise it and harm falls but false refusals climb; lower it and the model is more helpful but less safe.