harris corner detector

A corner is a point where two edges meet, and it is the most reliable kind of keypoint because it is pinned down in two directions at once. The Harris corner detector, published by Chris Harris and Mike Stephens in 1988, formalizes a simple intuition: slide a small window over the image and see how much the pixel contents inside it change. Over a flat region, sliding the window in any direction changes almost nothing. Along an edge, sliding parallel to the edge changes nothing (you cannot tell where you are along it), but sliding across it changes a lot. At a corner, sliding in any direction changes the contents a lot. A corner is precisely a location where appearance changes strongly under every shift.

To make this measurable, consider the weighted sum of squared intensity differences when the window is shifted by a small amount. Expanding this with a first-order Taylor approximation shows that the change is governed by a 2-by-2 matrix called the structure tensor (or second-moment matrix), formed by summing, over the window, the outer products of the image gradient with itself, weighted by a Gaussian. In words, the matrix accumulates the squared horizontal gradient, the squared vertical gradient, and the product of horizontal and vertical gradients across the window. Its two eigenvalues measure how strongly intensity varies along the two principal directions: two small eigenvalues mean a flat region, one large and one small mean an edge, and two large eigenvalues mean a corner.

Computing eigenvalues at every pixel was expensive in 1988, so Harris and Stephens defined a corner response that avoids it: the determinant of the matrix minus a small constant times the square of its trace. The determinant equals the product of the eigenvalues and the trace equals their sum, so this response is large and positive only when both eigenvalues are large (a corner), negative near edges, and small in flat regions. The empirical constant, usually between 0.04 and 0.06, controls sensitivity. Corners are then the pixels where this response is both above a threshold and a local maximum (non-maximum suppression). The closely related Shi-Tomasi detector ('Good Features to Track') instead uses the smaller eigenvalue directly, which is often more stable for tracking.

Because the structure tensor is built from gradients and summed over a circular Gaussian window, the Harris response is invariant to rotation (rotating the image rotates the eigenvectors but not the eigenvalues) and to additive brightness changes, and it degrades gracefully with noise. Its key weakness is that it is not scale invariant: a corner at one zoom level looks like a smooth curve when you zoom in far enough, so the same fixed window finds different points at different scales. This limitation directly motivated scale-space methods and the Harris-Laplace and SIFT detectors, which search across scales as well as positions.