linear discriminant analysis
/ LIN-ee-ur dih-SKRIM-uh-nunt uh-NAL-uh-sis /
Linear discriminant analysis solves a neat geometry problem: if you have to flatten labeled data down onto a single line, which direction should you flatten it along so the groups stay as separated as possible? Imagine apples and oranges plotted by weight and color, sitting in two clouds on a 2-D page. Cast their shadows onto the wrong line and the two clouds smear together; cast them onto the line LDA picks and the apple-shadows and orange-shadows land in two neat, well-spaced bunches you could split with a single mark.
It does this by balancing two goals at once: push the centers of the different groups as far apart as possible along the line, while keeping each group's own spread as tight as possible. The ideal direction maximizes the gap between groups relative to the scatter within them. Unlike PCA, which hunts for the directions of greatest overall variation and ignores the labels entirely, LDA uses the labels — it deliberately seeks the direction that best tells the classes apart. It can be read as either a way to reduce dimensions or as a classifier in its own right.
Its strengths are speed, simplicity, and a tidy interpretation; it's a strong baseline and works gracefully even with limited data because it makes strong assumptions to fill the gaps. And those assumptions are its honesty footnote: LDA presumes each group is shaped like a bell curve and that all groups share roughly the same spread and shape. When that's roughly true it's excellent; when groups have very different spreads or curved boundaries, it gets outclassed by more flexible methods. Note the unlucky clash of acronyms — this LDA has nothing to do with Latent Dirichlet Allocation, a topic-modeling method that shares the same initials.
Two iris species plotted by petal length and width form two overlapping ovals. PCA might pick the direction of biggest overall spread — which happens to smear the species together. LDA instead picks the diagonal that yanks the two species' averages apart; project onto it, and the two species fall into cleanly separated bands.
PCA ignores labels and maximizes spread; LDA uses labels to maximize separation.
The key contrast: PCA is unsupervised and maximizes overall variance; LDA is supervised and maximizes between-class separation. It assumes bell-shaped, equally-spread groups, so it shines when that holds and falters when it doesn't. (Don't confuse it with Latent Dirichlet Allocation, an unrelated topic model with the same initials.)