structure from motion

Structure from motion is the trick of recovering the 3D shape of a scene, plus where the camera was for each shot, using nothing but a set of ordinary 2D photos taken from different spots as the camera moved around. A single flat photo throws away depth — you cannot tell from one picture how far away anything is. But move the camera a few steps and snap again, and nearby things shift in the frame more than distant things, exactly the way a fence post near the road races past your car window while the far mountains barely budge. Structure from motion mines that shifting to reverse-engineer both the layout of the world and the path the camera took.

It works in two interlocking parts whose names give it its title. First it finds the same physical points across many photos — a particular corner of a window, the same crack in the pavement — by spotting distinctive features in each image and matching them. Then it solves a giant puzzle: there is essentially one consistent set of camera positions and one consistent 3D placement of those points that could have produced exactly the matches seen in every photo. Solving that puzzle gives you the structure (the 3D points) and the motion (the camera poses) at the same time, each helping pin down the other.

The payoff is that an ordinary moving camera becomes a 3D scanner. A drone circling a building turns its video into a 3D model; a phone walking around an object builds a printable mesh; a robot exploring a corridor recovers both the map and its own trajectory through it. The output is typically a sparse point cloud of the matched points together with the recovered camera path, which later steps can densify into full surfaces.