monocular depth estimation

Monocular depth estimation predicts a full depth map from a single photograph — no second view, no stereo baseline, no motion. This is striking because, strictly speaking, it is geometrically impossible: a single 2D image is consistent with infinitely many 3D scenes (a small nearby object and a large distant one can project to identical pixels). Yet humans glance at one photo and instantly sense what is near and far. We do it with learned priors — we know skies are far, faces are a certain size, parallel railway tracks converge, and texture grows finer with distance. Monocular depth networks learn exactly these priors from data.

Because the geometry is fundamentally ambiguous, the honest output is usually not metric. A single image cannot fix the overall scale (the small-object-versus-large-object ambiguity), so the best general models predict relative depth or, more precisely, scale-and-shift-invariant depth: the ordering and proportions of distances are correct, but converting to meters requires an unknown multiply and add. Models trained this way (MiDaS, DPT) generalize remarkably across cameras and scenes precisely because they sidestep absolute scale. Metric monocular depth (ZoeDepth, Metric3D) is harder and usually needs camera intrinsics or domain-specific training to anchor the scale.

The modern toolkit is deep and learning-driven. Early methods were CNN encoder-decoders trained on RGB-D datasets; the field jumped with transformer backbones (DPT uses a Vision Transformer) and with training on enormous mixed datasets. Depth Anything scaled this idea, training on millions of labeled and pseudo-labeled images to produce a robust foundation model for relative depth. A newer line treats depth as a generation problem: diffusion-based estimators like Marigold repurpose a pretrained image diffusion model to 'denoise' a depth map, inheriting strong scene priors and producing crisp, detailed results from very little depth training data.

Monocular depth is now everywhere because it needs only an ordinary camera: phone portrait-mode background blur, AR object placement, image-to-3D and novel-view pipelines, robotics where stereo is impractical, and as a prior that initializes or regularizes NeRF and Gaussian-splatting reconstructions. Its limitation is exactly its premise — without geometric evidence it can be confidently wrong on unfamiliar scenes, reflective or transparent surfaces, and anything that defies the learned size and layout priors.