self-supervised learning
/ self-soo-per-VIZED LER-ning /
Cover up a word in a sentence — "the cat sat on the ___" — and ask a learner to fill it in. To do that well, it has to figure out grammar, meaning, and the way the world works, even though no human ever wrote down a single answer. That is the trick of self-supervised learning: the data quietly supervises itself. The machine hides part of its own input and trains to recover it, manufacturing endless practice problems out of raw data.
Technically, self-supervised learning is a clever way to do supervised-style training on unlabeled data. You define a "pretext" task — predict the masked word, guess the next frame of a video, tell whether two crops came from the same photo — where the correct answer can be read straight off the data itself. Because the labels are free and automatic, you can train on staggering amounts of text, images, or audio. The model isn't really trying to be good at filling blanks; that chore just forces it to build rich internal representations that transfer to the tasks you actually care about.
This is the engine behind today's large language models and many vision systems: pre-train on oceans of unlabeled data with a self-supervised goal, then fine-tune on a small labeled set for the real job. The honest limits: designing a good pretext task is an art, pre-training is enormously expensive in compute and energy, and a model can become fluent at the pretext while still absorbing the biases and errors sitting in its raw training data.
Masked language modeling: take "Paris is the capital of [MASK]," delete "France," and train the model to predict it. Repeat across billions of sentences. No human labeled anything — the deleted word is its own answer — yet the model emerges knowing a great deal about language and facts.
The hidden word is the free, automatic label.
Self-supervised learning is a subtype of unsupervised learning: both use unlabeled data, but self-supervision manufactures its own answer keys from the data, turning the problem back into ordinary supervised training.