Evaluation, Robustness & Ethics

classification accuracy

Accuracy is the most intuitive way to grade a classifier: out of every image you showed it, what fraction did it label correctly? If a model sees 1,000 test images and gets 920 right, its accuracy is 920/1000 = 92%. It is a single number between 0 and 1 (often written as a percentage) that answers the plain question 'how often is it right?'. For ImageNet-style benchmarks you will also see top-5 accuracy, which counts a prediction as correct if the true class is among the model's five highest-scoring guesses — a gentler measure useful when many classes look alike.

Precisely, for a test set of N examples, accuracy is the number of examples whose predicted label equals the true label, divided by N. In confusion-matrix terms for binary problems it is (TP + TN) / (TP + TN + FP + FN), where TP/TN are correctly identified positives/negatives and FP/FN are the two kinds of mistakes. Accuracy weights every example equally and every error equally — getting a 'cat' wrong costs exactly as much as getting a 'truck' wrong, and a missed tumor costs the same as a false alarm.

That equal weighting is exactly where accuracy misleads. Under class imbalance — when one class dominates the data — a model can score high accuracy while being useless. If 99% of medical scans are healthy, a lazy model that always predicts 'healthy' achieves 99% accuracy yet catches zero disease. This is the 'accuracy paradox': the headline number looks excellent precisely because the rare, important class is being ignored. Accuracy also hides which class is failing and treats a confident wrong answer the same as a barely-wrong one.

When classes are imbalanced or errors have different costs, report richer metrics instead of (or alongside) accuracy: per-class accuracy and the full confusion matrix, balanced accuracy (the average of per-class recall, which re-weights rare classes up), precision and recall with the F1 score, ROC-AUC or precision–recall AUC for threshold-independent ranking quality, and cost-weighted metrics when a false negative is far worse than a false positive. Accuracy remains a fine summary for roughly balanced problems with symmetric error costs — just never trust it alone on skewed data.

Fraud-screening dataset with 10,000 images, 100 of them fraudulent (1%). A model that always says 'legitimate' scores 9,900/10,000 = 99% accuracy but its recall on fraud is 0/100 = 0% — high accuracy, total failure on the only class that matters.

A single accuracy figure is only comparable across models if they were evaluated on the identical test set and label set; 'top-1 on ImageNet' and 'accuracy on a rebalanced subset' are not the same yardstick.

Also called
accuracytop-1 accuracytop-5 accuracy