correlation filter tracking

Correlation-filter tracking learns a small filter that, when slid (correlated) across the search region, produces a sharp peak exactly at the target and near-zero everywhere else. The trick that makes it blazingly fast is to do the sliding in the Fourier domain, where correlation becomes a single element-wise multiplication. These trackers were a watershed because they run at hundreds of frames per second on a CPU while staying accurate.

The foundational MOSSE filter (2010) learns a filter h whose correlation output best matches a desired response, usually a 2D Gaussian peaked at the target center, by minimizing squared error; this least-squares problem has a closed form that is solved per-frame directly in the frequency domain. The deeper insight (Henriques' KCF, 2014) is that the set of all cyclic shifts of a base patch forms a circulant matrix, which is diagonalized by the Discrete Fourier Transform. This means the tracker is implicitly trained on thousands of shifted negative examples for almost free, and the ridge-regression solution reduces to elementwise division of Fourier transforms, even when a kernel (e.g., Gaussian or polynomial) is used to make the classifier nonlinear.

From this core, a family grew: KCF added multi-channel HOG features for far better accuracy; DSST and SAMF added explicit scale estimation (the base filter handles only translation); and later trackers replaced hand-crafted features with deep CNN features (C-COT, ECO) to top the benchmarks at the cost of speed. A built-in caveat is the boundary effect: the circulant/periodic assumption means the training samples wrap around the patch edges, creating artifacts and limiting the search range, which spatially regularized variants (SRDCF) and background-aware filters (BACF) were designed to fix.