optical flow

Optical flow is the pattern of apparent movement you see when you watch one video frame turn into the next. Picture two photos taken a fraction of a second apart: almost everything is in the same place, but a few patches of the image have shifted a little — a passing car slid to the right, a hand drifted upward. Optical flow draws a tiny arrow on every pixel showing which way and how far it seems to have moved between the two frames. Stack all those arrows together and you get a field of motion, a map of how the whole picture is flowing.

The word apparent matters: optical flow only knows that pixels moved, not why. The same rightward arrows could mean the car drove right, or the camera panned left, or both. A computer estimates this flow by assuming a small bright spot keeps roughly the same brightness from one frame to the next, then asking, for each patch, where did that brightness go? Matching the patches gives the arrows. It works beautifully when motion is small and smooth, and struggles on blank walls (no texture to track) or when things move too fast between frames.

For a robot this flowing field of arrows is a rich, cheap clue about motion using nothing but a plain camera. If the whole field streams outward from a central point, the robot is moving forward (everything rushes past faster at the edges, like the view through a car windscreen); and as the robot moves through the world a nearby object produces big arrows while the far background barely shifts, hinting at distance — though that depth cue only holds while the camera travels, not when it merely turns in place, since a pure rotation moves near and far alike. A patch of arrows running against the grain flags something that is moving on its own. Robots use optical flow to hold a steady hover, dodge oncoming obstacles, and gauge their own speed — much as a bee judges its flight by the world streaming past its eyes.