depth estimation

Depth estimation is the job of figuring out how far away each point in a view is from the camera — turning a flat image, which by itself has no sense of distance, into knowledge of near and far. The output is usually a depth map: an image where every pixel stores a distance instead of a color, with brightness standing in for how close (or, by convention, how far) each point is. A robot that knows depth can tell that the doorway is two meters ahead and the chair is half a meter to its left, which is what it takes to move and reach without bumping into things.

There is no single way to get depth; instead there is a toolbox of methods, each with trade-offs. Stereo cameras infer depth from the disparity between two side-by-side views. Structured light projects a known pattern of dots or stripes onto the scene and reads how the pattern shifts and bends over surfaces to compute distance — this is how many indoor depth cameras work. Time-of-flight sensors and lidar shoot out light and measure how long it takes to bounce back. And learning-based methods use a neural network that has studied millions of images to guess depth from a single ordinary photo, leaning on cues a human also uses, like the fact that things shrink with distance and that texture gets finer far away.

Each approach has a sweet spot and a weakness. Stereo and single-image learning struggle on blank, textureless surfaces; structured light and time-of-flight can be washed out by bright sunlight; lidar is accurate and long-range but expensive and gives sparse points rather than a dense image. Real robots therefore often combine several sources — fusing them so the strengths of one cover the blind spots of another — to build a depth picture reliable enough to act on.