feature scaling
/ FEE-cher SKAY-ling /
Feature scaling is the act of resizing each numeric feature so they all speak in comparable units. Think of converting every athlete's height-and-weight into the same scale before ranking them, so that pounds don't automatically outweigh inches just because the numbers happen to be bigger. Normalization and standardization are the two most common scaling recipes.
Concretely, a feature like income (tens of thousands) and a feature like number of children (zero to a handful) live on wildly different scales. Without scaling, any method that measures distance or adds up weighted features will be dominated by the large-numbered feature simply because of its magnitude. Scaling rescales each feature — for instance into a 0-to-1 range, or to a mean of 0 and standard deviation of 1 — so each gets a fair say. The scaling rule is always learned from the training set, then reused unchanged on new data.
Why it matters: distance-based methods (k-nearest neighbors, k-means), gradient-descent-trained models (most neural networks, logistic and linear regression), and regularized models all behave better and train faster when features are scaled. Some methods don't need it — tree-based models like random forests and gradient boosting split one feature at a time and are immune to scale — so scaling is a tool to apply when the method calls for it, not a ritual to perform blindly.
Finding a customer's nearest neighbors using income (in dollars) and age (in years), income's huge numbers swamp age entirely — everyone's "nearest neighbor" is just whoever has a similar salary. Scale both to 0–1 and age finally counts, so similarity reflects both traits.
Before scaling, dollars drown out years; after scaling, both traits count.
Scaling is not always needed: tree-based models (decision trees, random forests, gradient boosting) are scale-invariant. Knowing which methods care saves you from cargo-cult preprocessing.