motion estimation

Motion estimation is the umbrella task of inferring how things moved between frames, whether that is individual objects, the camera itself, or the scene as a whole. Optical flow is one answer (a dense per-pixel field), but motion can also be described much more coarsely and compactly, and the right representation depends entirely on the application: a video codec wants cheap motion that compresses well, a stabilizer wants the global camera motion, and a 3D reconstruction system wants motion consistent with scene geometry.

Block matching is the most widely deployed form, because it powers video compression. The frame is divided into blocks (e.g., 16×16), and for each block the encoder searches a reference frame for the most similar block, typically minimizing the sum of absolute differences (SAD) or sum of squared differences (SSD); the offset to the best match is the motion vector. MPEG, H.264/AVC, and H.265/HEVC all use this for inter-frame prediction: instead of storing a block, store a motion vector plus a small residual. Crucially, codec motion vectors are chosen to minimize bits, not to be physically accurate, so they can point to a visually similar but wrong location.

Parametric models compress motion further by assuming a whole region obeys one transformation: a 2-parameter translation, a 6-parameter affine model (translation, rotation, scale, shear), or an 8-parameter homography (a planar projective map, ideal for camera rotation or a flat surface). These few parameters are robustly fit, often with RANSAC to reject outliers, and are the backbone of video stabilization, panorama stitching, and global-motion compensation. Layered and segmentation-based models go further, assigning different parametric motions to different moving objects. The spectrum runs from dense and nonparametric (optical flow) to sparse and highly parametric (a single homography).