representation learning

Representation learning is the study of how to turn raw, high-dimensional inputs like pixels into a compact vector of numbers, an embedding, that captures the meaningful content rather than the surface noise. The goal is not to answer one specific question but to produce features that are useful for many questions you have not even asked yet. A good representation makes the easy structure of the world explicit: in a good embedding space, two photos of cats sit close together and far from photos of trucks, so that even a trivial classifier can separate them.

The objectives a representation is judged against are usually stated as desirable properties: linear separability (classes become separable by a simple linear boundary), transferability (the same frozen features help on new datasets and new tasks such as detection or segmentation), robustness (small irrelevant changes to the input barely move the embedding), and sometimes disentanglement (different coordinates capture different factors of variation like pose, colour, identity). These are the deep reasons we prefer learned features over hand-crafted ones like SIFT or HOG.

Evaluation is what makes representation learning a science rather than a feeling. The standard probes are: train a single linear layer on frozen features (the linear evaluation protocol); do k-nearest-neighbour classification directly in embedding space; fine-tune the whole network on a small labelled set; or transfer to a different downstream task entirely. Crucially, no single number tells the whole story, a method can have a mediocre linear probe yet excellent fine-tuning (this is exactly the MAE versus SimCLR contrast), so practitioners report several metrics across several datasets.