camera calibration

Camera calibration is the process of measuring how your specific camera turns 3D rays of light into 2D pixels. Every geometric algorithm in 3D vision — triangulation, stereo, structure-from-motion — assumes a precise model of that projection. Calibration estimates the model's parameters so that you can convert freely between a pixel and the ray it came from. Without it, you have pictures; with it, you have measuring instruments.

The parameters split into two groups. Intrinsics describe the camera's internal optics, independent of where it is: the focal length (in pixels, how strongly the lens magnifies), the principal point (the pixel where the optical axis hits the sensor, usually near but not exactly at the image center), and sometimes a skew term. Together these form the 3×3 intrinsic matrix K. Separately, lens distortion captures how real lenses bend straight lines — radial distortion makes the image bulge (barrel) or pinch (pincushion), and tangential distortion comes from a lens not perfectly parallel to the sensor. Distortion is modeled by a few polynomial coefficients and must be corrected before any straight-line geometry applies.

The standard procedure is Zhang's method: photograph a planar target with a precisely known pattern — a checkerboard or a grid of circles — from many angles. Because you know the real-world spacing of the corners and can detect them to sub-pixel accuracy in each image, you get a rich set of 3D-to-2D correspondences. The algorithm first solves a linear system for an initial guess of the intrinsics and each board's pose, then refines everything by nonlinear minimization of reprojection error (the same bundle-adjustment idea). The output is K plus the distortion coefficients; the per-image board poses (extrinsics) are a useful byproduct.

Calibration is what makes geometry metric and clean. Knowing K lets you upgrade an uncalibrated fundamental matrix to an essential matrix and thus recover true relative pose. Knowing distortion lets you undistort images so that straight scene edges become straight pixels and epipolar lines become exact. For stereo, calibrating the two cameras together (including their relative pose) enables rectification — the warp that puts corresponding points on the same scanline. Skip or sloppily perform calibration and every downstream 3D result inherits a systematic, hard-to-diagnose bias.