edge detection

An edge is a place in the image where brightness changes sharply, and such places usually mark something meaningful in the world: the boundary of an object against its background, a change of material or texture, a cast shadow, or a fold in a surface. If you trace the outline of a coffee mug with a pencil, you are intuitively doing edge detection, finding the curves where the mug's dark surface meets the lighter table. The goal of automatic edge detection is to convert a grey-scale image into a sparse map of these intensity discontinuities, which dramatically compresses the image while keeping most of its perceptually important structure.

The mathematical handle on a 'sharp change' is the derivative. Treating the image as a function of two coordinates that returns brightness, the gradient is the vector of its partial derivatives in the horizontal and vertical directions; in plain words, it points in the direction of fastest increase of brightness, and its length (the gradient magnitude) measures how steep that increase is. An edge is where the gradient magnitude is large. In practice the derivatives are estimated with small convolution masks such as the Sobel or Prewitt operators (which combine a difference in one direction with a little smoothing in the other), and pixels whose gradient magnitude exceeds a threshold are declared edges. The gradient direction, perpendicular to the edge, tells you the edge's local orientation.

There is a second, complementary family based on the second derivative. Where the first derivative peaks, the second derivative crosses zero, so edges can be found as zero-crossings of the Laplacian (the sum of the second partial derivatives). Because differentiation amplifies noise, the image is first smoothed with a Gaussian; the combined operator is the Laplacian of Gaussian (LoG), and the classic Marr-Hildreth method finds edges as its zero-crossings. Zero-crossing methods give closed, thin contours but can hallucinate edges in smooth regions, so the response is usually checked against a minimum gradient strength.

Every edge detector lives on a trade-off governed by the smoothing scale. Too little smoothing and noise produces spurious edges; too much and real but faint edges vanish and corners get rounded. This is why scale matters and why multi-scale edge detection exists. The practical gold standard for decades has been the Canny detector, which carefully orchestrates smoothing, gradient computation, thinning, and thresholding. Modern data-driven detectors such as Holistically-Nested Edge Detection (HED) and successors learn to predict edges with convolutional networks and align much better with human-drawn object boundaries, at the cost of needing training data and producing thicker, probabilistic edge maps.