3D Vision & Geometry

epipolar geometry

Imagine two cameras photographing the same scene. Pick a single point on an object — say the tip of someone's nose. It lands at one specific pixel in the left photo. Where can that same nose-tip appear in the right photo? Naively you might think it could be anywhere, so finding the match means searching the whole right image. Epipolar geometry is the beautiful fact that, given only the relative placement of the two cameras, the match is forced to lie on a single known line in the other image. This collapses a two-dimensional search into a one-dimensional one, which is the cornerstone of nearly every stereo and multi-view algorithm.

The reason is pure geometry. The 3D point, together with the two camera centers (the tiny pinholes where light converges in each camera), defines a plane called the epipolar plane. The left camera sees the point along a ray; that ray, projected into the right image, becomes a line — the epipolar line. The true match must sit somewhere on that ray in 3D, so its image must sit somewhere on that projected line in 2D. Every 3D point you could imagine along the left ray maps to a different position along the same right-image line, which is exactly why depth (how far along the ray) and image position on the line are two ways of saying the same thing.

Two recurring objects make the picture precise. The baseline is the line segment joining the two camera centers. Where the baseline pierces each image plane is called the epipole: the epipole in the right image is literally the projected position of the left camera center, and vice versa. All epipolar lines in an image pass through that image's epipole, like spokes through a hub. When the two cameras are arranged with parallel image planes and aligned rows (the rectified setup used in stereo rigs), the epipoles go to infinity and every epipolar line becomes a horizontal scanline, so matching reduces to scanning along the same row — the simplest and most common case in practice.

Epipolar geometry is captured algebraically by the fundamental matrix (uncalibrated cameras, pixel coordinates) and the essential matrix (calibrated cameras, normalized ray coordinates). It is purely a property of the two cameras' relative pose and internal optics — it does not depend on the scene at all. That is what makes it so powerful: estimate it once from a handful of correspondences, and it constrains every other correspondence for free.

Two photos of a desk taken a step apart: click a stapler corner in the left image, and the software draws a line across the right image. The matching corner is guaranteed to lie on that line — so the matcher only scans the line, not the whole 640×480 image, cutting roughly 300,000 candidate pixels down to a few hundred.

A common misconception is that the epipolar line tells you where the match is. It only tells you the line the match lies on; you still need photometric matching (or a learned matcher) to find the exact position along that line, and that position is what encodes depth.

Also called
epipolar constraint核線幾何