autoencoder

An autoencoder is a neural network trained to copy its input to its output — but through a deliberate bottleneck that forces it to throw most of the data away and keep only what matters. Think of it as learning to summarize an image into a short "code" and then reconstruct the picture from that summary. Because the network must rebuild the image from the squeezed-down code, it is pressured to discover an efficient, meaningful representation rather than memorizing raw pixels.

An autoencoder has two parts. The encoder, a function f, maps an input image x to a latent code z = f(x), usually of much lower dimension than x — this is the bottleneck. The decoder, a function g, maps the code back to a reconstruction x̂ = g(z). Both are trained jointly to minimize a reconstruction loss, for example the mean squared error ||x − x̂||² (squared per-pixel difference) or a per-pixel cross-entropy, averaged over the training set. No labels are needed — the target is the input itself — so this is self-supervised, unsupervised learning.

The bottleneck is the key regularizer: if z had as many dimensions as x with no constraint, the network could simply learn the identity map and learn nothing useful. Variants impose other constraints instead of, or in addition to, a narrow bottleneck. Denoising autoencoders corrupt the input and ask the network to restore the clean image; sparse autoencoders penalize the number of active code units; contractive autoencoders penalize the code's sensitivity to input changes. Autoencoders are used for dimensionality reduction (a nonlinear generalization of PCA), representation learning and pretraining, anomaly detection (high reconstruction error flags outliers), and image denoising.