image keypoint

Imagine you and a friend each photograph the same building from slightly different angles, then try to figure out which pixel in your photo corresponds to which pixel in theirs. Comparing every pixel is hopeless, because a flat blue patch of sky looks identical to a thousand other sky pixels. What you want instead are a few distinctive landmarks, like the tip of a flagpole or the corner of a window, that you could confidently point to in both photos and say 'that is the same physical spot.' An image keypoint is exactly such a landmark: a small, distinctive, repeatable location in an image that can serve as a stable anchor for matching, tracking, and reconstruction.

More precisely, a keypoint is a point (often with an associated scale and orientation, and sometimes an elliptical region) selected by a detector that responds strongly to local image structure such as corners or blobs. A good keypoint has four properties. Repeatability: it is found at the same physical point even when the image undergoes rotation, scale change, viewpoint shift, or illumination change. Distinctiveness: its surrounding patch is rich enough to tell it apart from other keypoints. Locality: it depends only on a small neighbourhood, so it survives occlusion and clutter. Accuracy: its position can be localized precisely, often to sub-pixel precision.

Keypoints are only half of the story. The detector decides where the interesting points are; a separate descriptor (SIFT, SURF, ORB, and so on) then encodes the appearance of the patch around each keypoint into a vector or binary string so that two keypoints can be compared numerically. This detect-then-describe pipeline is the backbone of classical structure-from-motion, visual SLAM, panorama stitching, and object recognition. Flat regions and straight edges make poor keypoints: a flat region is ambiguous in every direction, and a point on a straight edge can slide along the edge without changing appearance (the aperture problem), so detectors deliberately seek corners and blobs where appearance changes in two independent directions.

Modern deep learning has produced learned keypoint detectors and descriptors such as SuperPoint, D2-Net, R2D2, and DISK, often trained jointly and matched with learned matchers like SuperGlue or LightGlue. These can be far more robust under extreme viewpoint and illumination change than hand-crafted detectors, and end-to-end systems sometimes bypass explicit keypoints entirely. Even so, the keypoint abstraction remains central because it is sparse, interpretable, and geometrically meaningful, which is exactly what downstream geometry (triangulation, pose estimation) needs.