lenet
LeNet is the architecture that essentially invented the convolutional neural network as we use it today. Developed by Yann LeCun and colleagues across the late 1980s and refined into the famous LeNet-5 in 1998, it was built to read handwritten digits on bank cheques and postal envelopes. The core insight was that you should not flatten a 28x28 image into a flat list of 784 numbers and feed it to a generic fully-connected network, because that throws away the fact that nearby pixels are related and that a digit looks the same wherever it appears on the page. Instead, LeNet slides small learnable filters across the image (convolution), shrinks the result while keeping the strongest responses (pooling, called subsampling at the time), and only at the very end uses ordinary fully-connected layers to make the final class decision.
This conv-then-pool-then-fully-connected template is the genre-defining pattern that every later CNN inherited. LeNet-5 specifically stacks: a convolutional layer that produces several feature maps, an average-pooling (subsampling) layer that halves the spatial size, another convolution, another pooling, and then two or three fully-connected layers ending in a 10-way output for the digits 0 through 9. It used tanh and sigmoid squashing nonlinearities (ReLU had not yet become standard), average pooling rather than max pooling, and was trained end-to-end by backpropagation. By the standards of today it is tiny, roughly 60,000 parameters, yet it achieved error rates on MNIST below one percent and was deployed commercially to process a large fraction of US cheques in the 1990s.
What makes LeNet historically pivotal is that it demonstrated three durable ideas at once: local receptive fields (each neuron looks only at a small patch), weight sharing (the same filter is reused at every position, which is what makes the network translation-aware and dramatically cuts the parameter count), and a hierarchy of features (early layers detect edges and strokes, later layers assemble them into digit-like shapes). These three ideas are exactly what AlexNet, VGG, and ResNet would later scale up by orders of magnitude. LeNet proved the principle; it just had to wait roughly fifteen years for large labelled datasets and GPUs to let the principle reach photographs.
Worked sketch: a 32x32 grayscale digit enters LeNet-5; conv1 (six 5x5 filters) yields six 28x28 maps; 2x2 subsampling gives six 14x14 maps; conv2 (sixteen 5x5 filters) gives sixteen 10x10 maps; subsampling gives sixteen 5x5 maps; these 400 values flow through fully-connected layers of 120 then 84 units into 10 outputs, one per digit.
Do not assume LeNet looks like a modern CNN in every detail. It used average pooling and tanh/sigmoid, not max pooling and ReLU, and its second convolutional layer connected to only a hand-picked subset of the previous feature maps to save computation, an engineering compromise that later full connectivity made unnecessary.