dense optical flow

Dense optical flow estimates a motion vector at every single pixel, producing a full flow field rather than vectors at a sparse set of corners. The result is usually visualized by mapping flow direction to hue and flow magnitude to saturation, so a moving object lights up as a colored blob while the static background stays gray. Dense flow is what downstream tasks like video segmentation, frame interpolation, and motion-based action recognition actually consume.

Classical dense methods add a regularizer to defeat the aperture problem at flat and edge pixels. Horn-Schunck (1981) is variational: it minimizes brightness-constancy error plus a global smoothness term that penalizes large spatial changes in (u, v), spreading reliable corner estimates into ambiguous regions. Farneback (2003), the standard fast classical method in OpenCV, models each neighborhood as a quadratic polynomial in the pixel coordinates and solves for the displacement that maps one frame's local polynomial onto the next; it runs at interactive rates and handles moderate motion well.

Learned methods now dominate accuracy. FlowNet (2015) was the first end-to-end CNN to regress flow, trained on synthetic data (FlyingChairs) because dense ground-truth flow is nearly impossible to label by hand. The key architectural ideas matured into PWC-Net (a feature pyramid, warping the second frame toward the first at each level, and a cost volume of feature correlations) and then RAFT (2020), which builds a 4D all-pairs correlation volume between every pair of pixels and runs a recurrent GRU that iteratively refines a single high-resolution flow field. RAFT and its successors (GMA, FlowFormer) are the current strong baselines. Quality is measured by endpoint error (EPE), the average Euclidean distance between predicted and true flow vectors, on benchmarks like Sintel and KITTI.