autoencoder
/ AW-toh-en-koh-dur /
An autoencoder is a network that learns by trying to copy its input back out — but through a deliberate bottleneck that forces it to compress first. Imagine being asked to describe a photo to a friend in just ten words, and then having that friend redraw it from your description. To do well, your ten words must capture what truly matters — the gist — and throw away the clutter. An autoencoder plays both roles: an encoder squeezes the input down to a small summary, and a decoder tries to rebuild the original from that summary alone.
The clever part is what happens at the squeeze. The small summary in the middle — called the latent representation or bottleneck — is far too cramped to hold the input verbatim, so the network is pressured to keep only the essential structure and discard noise. And because the target output is just the input itself, no human labels are needed; the data supervises itself. This makes the autoencoder a classic example of unsupervised, representation-learning, where the goal is not a prediction but a good compressed description of the data.
What is it good for? The learned latent codes can be used to compress data, to remove noise (train it on noisy-in, clean-out pairs and it learns to scrub), or to spot anomalies (a fraud or a defect rebuilds poorly, because the network never learned to copy such oddities). The honest limits: a plain autoencoder is not a reliable generator of brand-new, realistic samples — its latent space is usually lumpy and full of gaps — and it can cheat if the bottleneck is too wide, simply passing the input through. Variational autoencoders and other variants address some of this, but for crisp image generation, GANs and diffusion models took the lead.
Feed a 784-pixel handwritten-digit image into an encoder that squeezes it to just 32 numbers; the decoder rebuilds the digit from those 32. Train on clean digits, then feed a noisy one — the decoder outputs a cleaned-up version, because it only knows how to draw real digits.
Squeeze to a tiny code, then rebuild — keep the essence, drop the noise.
An autoencoder's value is the squeezed code in the middle, not the copy it spits out — the copy is just the training pretext that forces a good code to form. If you make the bottleneck as wide as the input, it learns nothing and merely copies through.