JOVANA
Explore Library Glossary Getting Started Three Levels Fields How it works Mission
Join the mission
All guides

Shipping with Confidence: Evaluation-Driven Development, Guardrails, and LLMOps

LLMs are non-deterministic, so 'it worked when I tried it' is not enough. Build the eval first, then the feature.

Why classic software practices break

Ordinary code is deterministic: the same input gives the same output, so a unit test can assert exact equality. LLMs break that assumption. The same prompt can yield different wordings, and 'correct' is fuzzy — many phrasings are equally good. A change that fixes one case can silently break ten others. So you cannot eyeball a few examples and ship. You need evaluation as a continuous, measured practice, not a one-time check.

P(x_t \mid x_{<t}) = \mathrm{softmax}\!\left(\frac{z_t}{T}\right)

Sampling from this temperature-scaled distribution is why the same prompt can yield different wordings each run.

Evaluation-driven development

Evaluation-driven development flips the order: before you tune a prompt, you build the test set that defines success. Collect real (or realistic) inputs, write down what a good answer looks like, and turn that into a scorer. Now every prompt tweak, model swap, or RAG change is measured against the same yardstick. You stop arguing about whether output 'feels better' and start watching a number move.

  1. Build a golden set of 50–200 representative cases, including the nasty edge cases that broke you before.
  2. Choose scorers per case: exact match or schema-valid for structured output, reference-based metrics for extraction, and an LLM-as-judge for open-ended quality.
  3. Run the whole set on every change and track the score over time in version control, like any other test suite.
  4. Watch for contamination and judge bias — an LLM judge can be fooled, so spot-check it against human ratings.
For classification cases in your golden set, a confusion matrix shows exactly which scorer outcomes (TP/FP/FN/TN) shift on each change.

A two-by-two confusion matrix with true positives, false positives, false negatives, and true negatives.

Product evaluation: offline and online

Product evaluation has two halves. Offline evals run your golden set before release — fast, cheap, repeatable, and the gate for shipping. Online evals measure the live product: thumbs-up rates, task completion, escalation to humans, and A/B tests between prompt versions. Offline tells you whether the change is safe; online tells you whether users are actually better off. A model can win your benchmark and still annoy real people, so you need both.

Guardrails in production

Guardrails are the checks that wrap each model call so a bad input or output never reaches the user — or the user's data never reaches a tool that shouldn't see it. On the way in, you screen for prompt injection and out-of-scope requests. On the way out, you run content moderation, validate the output against your schema, and verify that claims are grounded in retrieved sources before you show them. Guardrails are layered defense, not a single filter: assume each layer is imperfect and stack them.

LLMOps: keeping it alive

LLMOps is the operational discipline once the app is live: versioning every prompt and model, logging full request/response traces for observability, monitoring quality, cost, and latency on real traffic, and being able to roll back instantly when a vendor silently updates a model and your eval score drops. The LLM you depend on can change underneath you, so treat the model as a versioned, monitored dependency — and keep the offline eval running on a schedule, not just at release.

LLMOps closes the loop: version, log, and monitor in production, then feed drift back into retraining and prompt updates.

The MLOps lifecycle diagram with a drift-detection feedback loop back to retraining.