mean-shift tracking

Mean-shift tracking describes the target by a histogram of its colors (or other features) and then, in each new frame, climbs to the nearest peak of a similarity surface by repeatedly nudging a search window toward its weighted center of mass. Picture the target as a 'color signature'; in the next frame you place the old window, ask each pixel 'how much do you look like the target?', and slide the window toward where the matching pixels cluster, repeating until it stops moving. No exhaustive search is needed, the window walks uphill to the target.

Precisely, mean shift is a mode-seeking procedure that performs gradient ascent on a kernel density estimate without ever computing a gradient explicitly. The target is a kernel-weighted color histogram (a smooth spatial kernel down-weights pixels near the window edge). A candidate window's histogram is compared to the target's via the Bhattacharyya coefficient, a similarity between probability distributions. Each pixel receives a weight equal to the square-root ratio of target-to-candidate histogram bins, and the new window center is the weighted mean of pixel positions. Iterating this mean shift provably converges to a local maximum of similarity.

Its strengths are speed (a handful of iterations, no full search) and robustness to rotation and non-rigid deformation, because a color histogram throws away spatial layout and so does not care how the object is bent or turned. The same property is its weakness: it ignores spatial structure, confuses targets with similarly colored background, and the basic algorithm cannot change scale. CAMShift (Continuously Adaptive Mean Shift) fixes the scale problem by resizing and reorienting the window from the moment (spread) of the back-projected histogram each frame, which is the version shipped in OpenCV and famously used for face/head tracking on hue.