capability vs. safety evaluation

Capability evaluation asks what a model can do — solve math, write code, answer questions — and you want those scores high. Safety evaluation asks what a model might do that is harmful — produce dangerous instructions, deceive, leak private data, comply with a jailbreak — and you want those scores low. They are different questions needing different tests, and a model can be excellent at one while alarming on the other.

The two are entangled in an uncomfortable way: rising capability often raises risk. A model good enough to write helpful code is also good enough to write malware; one that reasons well can reason its way around a guardrail. So you cannot read safety off a capability benchmark, and progress on one does not guarantee progress on the other. Safety evaluation also leans harder on adversarial testing — red-teaming, jailbreak suites, probing for deception — because you are searching for rare worst cases, not average performance.

Mature evaluation reports both side by side and resists collapsing them into one headline. A high capability score with an unmeasured safety profile is not a green light; the responsible question is not just how good is it, but how good is it and how could it go wrong.