sift descriptor

SIFT, the Scale-Invariant Feature Transform introduced by David Lowe in 1999 and detailed in 2004, was the breakthrough that made reliable image matching across large changes in scale, rotation, and viewpoint practical, and it dominated computer vision for more than a decade. The name covers a complete pipeline: detect keypoints, assign each an orientation and scale, and describe the local patch with a vector engineered to be repeatable. Its enduring lesson is that invariance is built in deliberately at each stage rather than hoped for.

Detection uses scale-space extrema of the Difference of Gaussians, which gives each keypoint a position and an intrinsic scale (so the descriptor will be computed over a patch whose size matches the feature, granting scale invariance). Candidate extrema are refined to sub-pixel accuracy by fitting a quadratic, and unstable points are discarded: low-contrast points are rejected, and points lying on edges (where one principal curvature dominates, detected via the ratio of Hessian eigenvalues) are rejected because they localize poorly. Each surviving keypoint is then assigned a dominant orientation from a histogram of local gradient directions, weighted by gradient magnitude; rotating the descriptor frame to this orientation is what grants rotation invariance.

The descriptor itself is the famous 128-dimensional vector. Around the keypoint, in a region whose size scales with the keypoint and whose axes are rotated to the keypoint's orientation, SIFT lays a 4-by-4 grid of subregions. In each subregion it builds a histogram of gradient orientations into 8 bins, weighted by gradient magnitude and by a Gaussian that down-weights gradients far from the centre. Stacking 4 times 4 times 8 gives 128 numbers. The vector is then normalized to unit length to cancel uniform contrast changes, its large entries are clipped (to limit the influence of a few strong gradients, for example from non-linear illumination), and it is renormalized. The use of orientation histograms rather than raw pixels is what makes SIFT tolerant of small geometric distortions and lighting changes.

In practice SIFT descriptors are matched by nearest neighbour in Euclidean distance, filtered by Lowe's ratio test (accept a match only if the closest neighbour is clearly closer than the second-closest), and the surviving matches feed geometric verification with RANSAC. SIFT is highly distinctive and remarkably robust, at the cost of being slower and more memory-hungry than binary descriptors like ORB. Once encumbered by a patent, SIFT became freely usable when the patent expired in 2020 and is again in mainline OpenCV. Although learned descriptors (HardNet, SOSNet) and end-to-end matchers (SuperPoint plus SuperGlue) now exceed it on hard benchmarks, SIFT remains a strong, interpretable baseline and is still widely used in structure-from-motion and 3D reconstruction.