fisherfaces

Fisherfaces is a face-recognition method that fixes the central blind spot of eigenfaces: eigenfaces compresses for total variation, but the loudest variation in face data is usually lighting and expression, not identity. Fisherfaces instead compresses for discrimination — it deliberately seeks the directions that pull different people apart while squeezing the images of the same person together. The intuition is a tug-of-war: project the faces onto a line (or a few directions) chosen so that the projected clusters for different identities are far from each other yet each cluster is tight.

This is linear discriminant analysis (LDA, here Fisher's criterion). You define two quantities from labeled training faces: the within-class scatter, which measures how spread out images of the same person are, and the between-class scatter, which measures how far apart the different people's average faces are. Fisherfaces finds the projection directions that maximize the ratio of between-class scatter to within-class scatter — push the class means apart, pull each class's members together. Because the data are very high-dimensional and the within-class scatter matrix would be singular (you have far fewer images than pixels), the standard recipe of Belhumeur, Hespanha, and Kriegman first reduces dimensionality with PCA (eigenfaces) and then applies LDA in that reduced space; the result is at most (number of classes − 1) discriminant directions. Recognition is again nearest-neighbor, but now in this discriminative subspace.

Empirically, Fisherfaces is markedly more robust to lighting and facial expression than eigenfaces, because by design it can learn to ignore the within-person nuisances that those create — that is the whole point of dividing by within-class scatter. The catch is that LDA is supervised and needs several examples per identity to estimate within-class scatter, struggles when there is only one image per person, can overfit if classes are tiny, and remains a purely linear projection. It is the canonical example of supervised dimensionality reduction in vision, and the lineage is clear: just as eigenfaces gave way to learned embeddings, modern deep face recognition (FaceNet's triplet loss, ArcFace's angular margin) pursues the very same goal — large between-identity separation and small within-identity spread — but in a learned nonlinear space.