Objects as parts on springs: the deformable part model
By the end of this track you have a recognition toolbox: templates and the colour-plus-pipeline view (Guide 1), descriptors fed to k-NN and SVMs and the bag-of-features trick (Guide 2), the Viola–Jones sliding-window face detector with Haar features and boosting (Guide 3), and the subspace face models of eigenfaces and Fisherfaces plus deformable face shapes (Guide 4). This final guide reaches the summit of the pre-deep-learning era and then walks you down the other side, into deep learning. We start with the single method that, for years, sat at the very top of object detection: the deformable part model, or DPM.
The picture to hold in your head is simple and physical. Imagine you want to find a person. You take one coarse, low-resolution root template that captures the whole figure as a blurry HOG silhouette — head over torso over legs — and you also keep several smaller, higher-resolution part templates: one for the head, one for a shoulder, one for each leg. Now connect each part to the root with a spring. The spring has a preferred (anchor) position, but it can stretch: the head can sit a little high, a leg can swing out. Detection asks two questions at once — does the image look like the root and the parts, AND did the parts stay near where the springs want them?
Every ingredient here is something you already know. The templates are made of HOG features — the gradient-orientation histograms introduced earlier in this track. Each template is scored by a learned linear filter, which is exactly an linear SVM weight vector dotted with the features. And to actually find an object in a photo, the whole model is slid across the image at many positions and scales — the sliding-window detection paradigm from Guide 3. DPM is not a new primitive; it is a clever assembly of the primitives this track already gave you, plus one new idea: the springs.
The DPM detection score: appearance reward minus deformation penalty.
Let us unpack this symbol by symbol, because it is the whole model in one line. The placements are p_0, p_1, …, p_n: p_0 is where you put the root, and p_1…p_n are where you put the n parts (each a position, and at a chosen resolution level). The first sum is the appearance reward. F_i is the i-th learned filter — F_0 is the root filter, F_1…F_n the part filters — and each is a weight vector trained like an SVM. φ(H, p_i) means 'pull out the HOG features from the image feature pyramid H at location p_i'. The dot product F_i · φ(H, p_i) is just the SVM filter response: a big positive number when the patch under that filter looks like what the filter was trained to like. Summing over i adds up how well the root and every part match. The second sum is the deformation penalty. φ_d(p_i − p_0) measures how far part i sits from its ideal anchor position relative to the root (in practice the squared horizontal and vertical displacements, dx, dx², dy, dy²), and d_i is a learned non-negative weight vector — the stiffness of that part's spring. So d_i · φ_d(...) grows as the part is dragged away from home. The minus sign is the key: the final score is 'how good everything looks' MINUS 'how much we had to bend the springs to make it fit'.
A tiny concrete example makes the trade-off vivid. Suppose the head filter fires strongly (say +5) but only if you place it 20 pixels above the root's anchor, while the spring's stiffness charges 0.01 per pixel². The deformation cost is 0.01 × 20² = 4, so the net contribution of the head is 5 − 4 = +1: still worth it, the model accepts a slightly high head. If instead matching that head needed a 40-pixel stretch, the cost would be 0.01 × 40² = 16, swamping the +5 reward — the model would rather not place the head there. That is exactly how DPM tolerates a person who is leaning, mid-stride, or slightly turned: parts can move to chase the real image evidence, but only as far as the appearance gain justifies the spring cost.
One last training subtlety earns DPM its place at the summit: latent-SVM learning. When you label training data you draw a box around the whole person, but you do NOT annotate where the head, shoulders and legs are — those part locations are hidden, or latent. DPM treats the best part placements as latent variables: during training it slides the parts to wherever they score highest for each example (an inner optimization), then updates the filters and spring weights with an SVM-style objective (an outer optimization), and repeats. The model effectively discovers, on its own, that 'there should be a part here, and it tends to sit there'. This is why DPM topped the PASCAL VOC detection challenge for several years in a row around 2008–2011 — it was the strongest general object detector in the world before deep nets arrived.
Voting for structure: the Hough transform
DPM learns what an object looks like from examples. Now we switch to a completely different question: detecting geometric structure that needs no training at all. The classic problem is this: an edge detector has given you a scattered cloud of edge points, and you want to find the straight lines hiding in that cloud — even though the lines have gaps (a dashed road marking), even though there is noise (stray edge pixels that belong to nothing). A naive approach of 'try fitting a line to every subset of points' explodes combinatorially. The Hough transform sidesteps this entirely with one beautiful idea: let every point vote.
Here is the voting idea in plain words. A single edge point does not know which line it belongs to — but it does know the (infinitely many) lines that could pass through it. So instead of committing, each edge point casts a vote for every line consistent with it. We tally these votes in an accumulator, one bin per possible line. A real line in the image has many edge points lying on it, and every one of them votes for that same line, so its bin accumulates a tall peak. Noise points scatter their votes thinly everywhere and never build a peak. Detection becomes 'find the bins with the most votes'. Gaps stop mattering, because we never need a continuous run of points — just enough of them, anywhere along the line, to agree.
The normal (ρ, θ) parameterization of a line — bounded and able to represent vertical lines.
This equation is how we name a line without the usual slope-intercept y = mx + c. Read it as: the point (x, y) lies on the line described by the pair (ρ, θ). Here (x, y) is an edge-point coordinate in the image; θ is the angle of the line's normal (the direction perpendicular to the line); and ρ is the perpendicular distance from the origin to the line. Every straight line in the plane corresponds to exactly one (ρ, θ) with θ in [0, 180°) and ρ a signed distance. Why bother, instead of y = mx + c? Because a vertical line has infinite slope — m blows up and the slope-intercept form cannot represent it at all. The (ρ, θ) form has no such blind spot: a vertical line is just θ = 0 with ρ equal to its x-position. And both parameters are bounded — θ within half a turn, ρ within the image diagonal — so a finite accumulator array can cover every possible line. As a quick check: the horizontal line y = 3 has a vertical normal, θ = 90°, giving ρ = x·0 + y·1 = 3, exactly its distance from the origin. The form behaves.
Now the magic move. Fix one edge point (x, y) and let θ sweep across all its values; ρ = x·cosθ + y·sinθ then traces a smooth sinusoid in the (ρ, θ) accumulator. So a single point in image space becomes a whole curve in parameter space — that curve IS the point casting its vote for every line through it. Take a second edge point and it draws its own sinusoid. Where two sinusoids cross, the (ρ, θ) there satisfies both points — i.e. it is the line that passes through both. A whole set of collinear points produces a bundle of sinusoids that all cross at one common (ρ, θ): that crossing is a tall peak in the accumulator. Peak-finding (scan the accumulator for local maxima above a vote threshold) is therefore the detection step, and each peak's coordinates (ρ, θ) hand you a detected line directly.
# Hough line voting (pseudocode)
# Inputs: edge_points = list of (x, y) from an edge detector
# Output: accumulator A indexed by (rho_bin, theta_bin)
A = zeros(num_rho_bins, num_theta_bins)
for (x, y) in edge_points:
for theta in range(0, 180): # sweep all line orientations
rho = x * cos(theta) + y * sin(theta)
r = quantize(rho) # map rho to its bin index
t = quantize(theta)
A[r, t] += 1 # this point votes for line (rho, theta)
# Detection = peaks in the accumulator
lines = []
for (r, t) in local_maxima(A):
if A[r, t] > vote_threshold: # enough points agreed -> a real line
lines.append((rho_of(r), theta_of(t)))Beyond lines: circles and the generalized Hough transform
Voting was never really about lines — it was about consensus in a parameter space. So let us change the shape. To detect circles, we just change what each point votes for and what the accumulator counts.
A circle has three parameters: centre (a, b) and radius r.
Read this as: the edge point (x, y) lies on the circle whose centre is (a, b) and whose radius is r. So a and b are the centre's coordinates and r is the radius. Compared with a line's two numbers (ρ, θ), a circle needs three numbers, which means the Hough transform accumulator is now a 3-D array A(a, b, r) — one bin for every candidate centre-and-radius. The voting logic is the same in spirit: each edge point votes for every (a, b, r) circle that could pass through it, and bins where many edge points agree light up as peaks. A coffee-cup rim photographed at one size produces a cluster of votes around one (a, b, r); the peak gives you its centre and radius at once.
There is a lovely efficiency trick, and it reuses something you already have. A good edge detector reports not just where an edge is but its gradient direction — and for a point on a circle, the centre always lies along the line normal to the edge, i.e. straight in (or out) along the gradient. So instead of letting an edge point vote for centres in every direction, you only vote along its gradient line, at each candidate radius. That collapses a 2-D fan of centre votes down to a 1-D ray per radius, slashing both the number of votes cast and the compute, and sharpening the peak (fewer stray votes muddy the accumulator). The same idea tames the cost of the 3-D accumulator that would otherwise be daunting.
Finally, the Generalized Hough Transform (GHT) frees voting from any equation at all. Suppose you want to detect an arbitrary rigid shape — a logo, a tool silhouette — for which there is no neat formula. In a training step you pick a single reference point inside the shape (say its centre) and walk around its boundary; for each boundary edge point you record the offset vector from that edge point to the reference point, and you file that offset in an R-table keyed by the edge point's gradient orientation. At detection time, each image edge point looks up its gradient orientation in the R-table, retrieves the stored offsets, and votes for the implied reference-point locations. Where the shape truly is, all these votes pile onto one spot. Add extra accumulator dimensions and the same scheme votes for scale and rotation too.
Reading letters: OCR as a complete classical pipeline
Now the capstone. Optical character recognition (OCR) — turning a picture of text into machine-readable characters — is the perfect finale because a real OCR system exercises nearly every idea in this whole track, composed into one shipping product. There is no single clever model here; there is a pipeline, each stage handing its output to the next, exactly the classical recognition pipeline mindset from Guide 1.
Diagram of an image-classification pipeline: an input image is turned into features and then assigned a class label.
- Detect and binarize: find text regions in the page and threshold the pixels into ink-vs-paper (black text on white), often after lighting correction so a shadow does not look like a stroke.
- Deskew: estimate the slight rotation of the scanned page and rotate it back so text lines are horizontal — frequently done with the Hough transform from the previous sections, voting for the dominant line angle.
- Segment: split the page into lines, lines into words, and words into individual character glyphs — using projection profiles, gaps, or connected components (groups of touching ink pixels).
- Normalize: rescale each glyph to a standard box and centre it, so an 'A' is compared at a canonical size and position regardless of the original font size.
- Extract features: turn each normalized glyph into a descriptor — pixel counts in zones, stroke directions, even a HOG of the glyph — the describe step from Guide 2.
- Classify: feed the descriptor to a k-NN or an SVM image classifier to decide which character it is — the decide step from Guide 2.
- Post-process: clean up the raw character string with a dictionary or language model, fixing classifier errors using context.
Look how many old friends reappear. For a fixed, known font, recognizing a glyph can be plain template matching (Guide 1) — overlay each candidate letter template and pick the best fit. Scanning a sliding window across a text line, classifying at each position, is exactly sliding-window detection (Guide 3) applied to characters. Connected components — flood-filling clusters of touching black pixels — are a fast way to isolate glyphs without any learning. The OCR pipeline is not a new theory; it is the entire toolbox of this track, snapped together for one job.
The hard part is a genuine chicken-and-egg: segmentation and recognition depend on each other. You cannot perfectly cut the word into letters until you know what the letters are, but you cannot recognize the letters until you have cut them out. Touching letters ('rn' looking like 'm'), broken strokes, and ligatures all make the cut ambiguous. Worse, glyphs are genuinely confusable in isolation: lowercase 'l' versus digit '1', capital 'O' versus zero '0', 'rn' versus 'm'. Context is what saves you — and that is the job of the post-processing language model.
The standard tool for that final correction is edit distance (Levenshtein distance): the minimum number of single-character edits needed to turn one string into another, where the three allowed edits are insertion, deletion, and substitution. If the classifier outputs the noisy string 'recognize' as 'rec0gnize', the dictionary word 'recognize' is just one substitution away (edit distance 1), while almost every other dictionary word is far further; so the correction picks 'recognize' as the nearest legal word. In a sentence the language model goes further, weighing which words are likely to follow which, so 'c0ld day' is corrected to 'cold day' and a confused 'l'/'1' is resolved by whether letters or digits are expected here. This is how classical OCR turns a stream of shaky per-character guesses into clean, correct text.
What classical recognition got right — and its ceiling
Step back and look at the whole track honestly. The classical era got an enormous amount right, and pretending otherwise would be bad history and bad engineering. Its methods are interpretable: you can literally render a Haar feature, an eigenface, a HOG template, or a DPM root filter and see what the system responds to — no mystery. They are data-efficient: template matching or a small SVM can work from a handful of examples, sometimes one. They are fast and light, which is why Viola–Jones ran on a 2001 webcam. And they are mathematically grounded — PCA, SVM margins, and Hough voting all rest on clear, analyzable principles.
And these methods did not retire — they are all around you. Template matching inspects circuit boards and pills on a manufacturing line, where the part is fixed and speed and reliability matter more than flexibility. The Hough transform finds lane lines in driver-assistance cameras and circles in industrial gauges. OCR is everywhere: cheque processing, licence-plate readers, document digitization, the 'scan to text' on your phone. When the problem is constrained and you need transparency, the classical recognition pipeline is often still the right tool.
But there was a real ceiling, and the field hit it. Every method here leans on hand-crafted features — a human decided that gradients (HOG), or Haar rectangles, or projection profiles were the right thing to measure for this particular problem. That hand-design is brittle: features tuned for upright frontal faces falter on profiles; HOG tuned for clean pedestrians struggles with heavy occlusion, odd lighting, motion blur, and the long tail of real-world variation. You could keep bolting on more parts, more cascade stages, more spatial bins via spatial pyramid matching — but returns diminished. The most telling symptom was the PASCAL VOC stagnation: around 2008–2011, DPM and its refinements topped the leaderboard, and yet the headline accuracy barely crept upward year over year. The community was polishing the same kind of model against a wall.
The bridge to deep learning
It is tempting to tell the story as a rupture — '2012, deep learning arrived, the old ways were swept away'. That is the wrong story. The truth is continuity: almost every classical idea in this track has a direct descendant living inside modern networks, often doing the very same job under a new name. Understanding the classical methods is the cheapest way to make the deep ones stop feeling like magic.
- Sliding window → convolution itself. Sliding one filter across an image and recording its response at every position is literally what a convolutional layer does; running a detector densely with no fixed crop is fully-convolutional detection.
- HOG and hand-crafted features → learned convolutional filters. HOG measures gradient orientations by hand; a CNN's first layers learn edge- and orientation-detectors that look strikingly similar — but trained from data, then stacked into ever more abstract features.
- Bag-of-features and spatial pyramids → pooling and global pooling. Aggregating local descriptors while keeping coarse spatial layout became max/average pooling and global average pooling inside the network.
- AdaBoost cascades and NMS → detection post-processing. The cascade's reject-early idea and non-maximum suppression for merging overlapping boxes are still standard glue in modern detectors.
- Eigenfaces and subspaces → learned embeddings. Projecting a face into a low-dimensional space where identity is easy to compare is exactly what a face-recognition network's embedding layer does — now learned, not from PCA.
- The hand-built recognition pipeline → one end-to-end learned pipeline. The detect → describe → classify → post-process stages of the classical recognition pipeline are fused into a single network trained jointly from pixels to answer.
So here is the master perspective to carry out of this track. In 2012 AlexNet did not invent a brand-new way of seeing and throw the old one away. It kept the deepest structural ideas — convolution as sliding template matching, pooling as aggregation, a layered pipeline from edges to objects — and changed exactly one thing: instead of a human hand-designing the features, the network learned them from data, end to end. The classical era spent decades carefully designing what to measure; the deep era automated that design. Everything you built up in these five guides is therefore not obsolete trivia — it is the conceptual skeleton that the deep methods put muscle on.
That is the bridge. From here the ladder crosses into the deep-learning tracks: convolutional networks proper, modern detectors and segmenters, and beyond. You will meet them not as strangers but as the grown-up forms of sliding windows, filters, voting, and pipelines you already understand. Welcome to the other side of the summit — the climb down is the start of a much bigger mountain.