scale space

The same object can appear large or small in an image depending on how far away it is, and features that are obvious at one size are invisible at another: the texture of tree bark is detail up close but disappears into a uniform brown from across a field. A scale space is the systematic answer to 'at what size should I look?' It is not a single processed image but an entire family of images, indexed by a scale parameter, in which fine detail is progressively removed by increasing blur. Reading the family from sharp to blurry is like stepping backward from the scene: small structures fade first, large structures persist.

Concretely, the Gaussian scale space of an image is obtained by convolving it with Gaussian filters of increasing standard deviation. The standard deviation is the scale: at scale zero you have the original image, and as it grows, the image is smoothed more and more. Crucially, Gaussian smoothing is the unique linear operation that builds a scale space satisfying natural axioms: increasing scale never creates new structures (no new local extrema appear out of nowhere as you blur), the representation behaves consistently under shifts and rotations, and combining two smoothing steps is equivalent to one larger step. These results, due to Witkin, Koenderink, Lindeberg and others, single out the Gaussian as the proper smoothing kernel rather than an arbitrary choice.

The payoff is scale-invariant feature detection. To find a feature regardless of its size in the image, search not only over spatial position but also over scale, and pick locations that are extreme in both at once; the scale at which the response peaks is the feature's intrinsic size. To make responses comparable across scales, derivative operators must be scale-normalized (for example multiply the Laplacian by scale squared), because raw derivatives shrink as the image is blurred. This is exactly what blob detectors and SIFT do: SIFT builds a discrete scale space organized into octaves (each octave covering a doubling of scale, with the image downsampled by half between octaves for efficiency) and detects keypoints as extrema of the Difference of Gaussians across that space.

Scale space is more than a SIFT implementation detail; it is a foundational principle of multi-resolution vision. The image pyramid, used everywhere from optical flow to modern feature-pyramid networks in object detectors, is a sampled scale space. Even convolutional networks implicitly build a learned multi-scale representation as receptive fields grow with depth. Understanding scale space explains why a single fixed-size filter cannot be scale invariant and why almost every robust feature pipeline processes the image at multiple resolutions.