depth estimation

Depth estimation is the task of assigning to every pixel a distance: how far the surface seen at that pixel is from the camera. The natural output is a depth map, an image whose values are meters (or some consistent unit) rather than colors. Depth is the missing third dimension that a flat photo throws away, and recovering it is the gateway to almost everything in 3D vision — reconstruction, robot navigation, augmented reality occlusion, autonomous driving, and 3D photography all rest on it.

There are two broad families. Active sensing measures depth physically: LiDAR fires laser pulses and times their return; time-of-flight cameras do the same with modulated light; structured-light sensors (like the original Kinect) project a known pattern and read its deformation. These give direct, metric depth but cost power, hardware, and often struggle outdoors or on shiny/dark surfaces. Passive sensing infers depth from ordinary images: stereo from two views via disparity, multi-view stereo from many views via photo-consistency, and monocular depth from a single image using learned priors about how the world looks.

A crucial distinction cuts across all methods: metric versus relative depth. Metric depth gives true distances in meters and requires either a calibrated sensor, a known baseline, or some absolute reference. Relative (or affine-invariant) depth gives only the ordering and proportions — this pixel is twice as far as that one — which is all a single uncalibrated image can honestly provide because of scale ambiguity. Many modern learned models predict relative or scale-and-shift-invariant depth precisely because it generalizes across cameras and scenes; recovering metric scale then needs extra information.

The state of the art mixes geometry and learning. Classical stereo and MVS give strong geometric depth where multiple views exist; learned models (MiDaS and DPT for relative depth, Depth Anything trained on enormous mixed datasets, Metric3D and ZoeDepth for metric monocular, and diffusion-based estimators like Marigold) supply remarkably robust depth even from a single frame. In practice systems fuse them: geometry where it is reliable, learned priors to fill occlusions, textureless regions, and the inherently ambiguous monocular case.