representation learning
/ rep-ruh-zen-TAY-shun LER-ning /
Before you can reason about a problem, you need a good way to describe it. A chess master doesn't see 64 separate squares; they see patterns — pins, forks, weak pawns — a representation that makes the right move obvious. Representation learning is about letting a machine discover, on its own, a good way to describe raw data, instead of having humans hand-craft those descriptions.
In older machine learning, people spent enormous effort on "feature engineering": deciding by hand which measurements to extract from raw data (edge counts, word frequencies) so a simple model could use them. Representation learning flips this: the model learns the useful internal description directly from the data. Deep neural networks do this layer by layer — early layers turn pixels into edges, later ones into shapes, then objects — building a hierarchy of increasingly meaningful features. The learned representation, often a compact vector called an embedding, places similar things near each other in a space where downstream tasks become easy.
This is arguably why deep learning took over: a single learned representation can serve many tasks, and it captures structure no human would have thought to hand-code. The honest qualifications: the representations are usually opaque — we can't easily read what each dimension "means" — and they faithfully encode whatever is in the data, biases included. A representation is only as good and as fair as the data and objective that shaped it, and "good" is relative to the task you ultimately care about.
Word embeddings turn each word into a vector. After training on lots of text, words used in similar ways land near each other — "king" sits close to "queen," "Paris" near "London." Nobody told the model what a king is; it learned a representation in which meaning becomes geometry, and arithmetic like king − man + woman ≈ queen even roughly holds.
A learned representation turns meaning into geometry.
Learned representations are powerful but opaque — you usually can't read what each dimension means — and they encode whatever the data carries, biases and all. The representation is only as fair as the data and objective that shaped it.