multilayer perceptron

A multilayer perceptron (MLP) is the original deep neural network: several layers of neurons stacked one after another, where every neuron in one layer connects to every neuron in the next, with a nonlinearity between layers. Information flows strictly forward (input layer, one or more hidden layers, output layer) with no loops. It is the baseline architecture you reach for before adding the structural tricks (convolution, attention) that specialize a network for images or sequences.

An L-layer MLP composes affine transformations with nonlinearities: h0 = x, then for each layer l, h_l = phi(W_l·h_(l-1) + b_l), and the output is read from the last layer (often with a task-specific final activation such as softmax). Each W_l is a weight matrix, each b_l a bias vector, and phi an elementwise activation such as ReLU. The hidden layers are the ones whose outputs you never directly observe; their width (number of neurons) and the network's depth (number of layers) are the main capacity knobs.

With even a single sufficiently wide hidden layer and a nonlinear phi, an MLP can approximate any continuous function on a bounded domain (the universal approximation theorem), so in principle an MLP could classify images. In practice it is a poor fit for raw pixels: flattening an image into a vector throws away spatial structure, and connecting every pixel to every hidden unit creates an enormous number of weights that overfit and ignore the fact that a cat is a cat wherever it appears. Convolutional networks fix exactly this by sharing weights over local patches.

Despite this, MLPs are everywhere inside modern vision systems, just not on raw pixels. The classification head on top of a CNN or ViT backbone is an MLP; every Transformer block contains a position-wise MLP (its feed-forward network) after attention; and architectures like MLP-Mixer (2021) showed that, given the right tokenization, MLPs alone can be competitive on image classification. The MLP is thus both the historical baseline and a live building block.