normalization and standardization
/ nor-muh-lih-ZAY-shun and stan-der-dih-ZAY-shun /
Suppose one column of your data is age (0 to 100) and another is annual income (0 to several hundred thousand). To a math-based model, income looks a thousand times bigger and shouts over age, just because of its units — not because it matters more. Normalization and standardization are two ways of putting all features onto a comparable scale so none drowns out the others.
Normalization usually means min-max scaling: squeeze each feature into a fixed range, typically 0 to 1, by mapping the smallest value to 0 and the largest to 1. Standardization means rescaling so each feature has a mean of 0 and a standard deviation of 1 — the so-called z-score, which says how many standard deviations a value sits above or below average. Normalization preserves the original shape but bounds the range; standardization centers the data and is more robust when the spread, not the bounds, is what matters.
Why it matters: many methods are sensitive to scale — gradient descent converges faster, distance-based methods like k-nearest neighbors and k-means behave sensibly, and regularization treats features fairly only when scales are comparable. The crucial discipline: compute the min, max, mean, and standard deviation only from the training data, then apply those same numbers to validation and test data. Re-deriving them on the test set leaks information and inflates your score.
Ages run 20–60 and incomes run 30,000–150,000. Min-max normalization maps a 40-year-old earning 90,000 to (0.5, 0.5). Standardization instead reports how unusual each value is: an age and income each exactly at the mean both become 0, regardless of their original units.
Min-max squeezes into 0–1; z-score recenters to mean 0, spread 1.
These terms are used loosely and people sometimes swap them. What is non-negotiable is fitting the scaler on training data only: an outlier or a peek at the test set's range will quietly distort the numbers everyone downstream relies on.