machine epsilon
/ EP-suh-lon /
Machine epsilon answers a blunt question: starting from 1.0, what is the smallest number you can add and still get an answer different from 1.0? Anything smaller simply rounds away and is lost. That threshold is a direct, tangible measure of how fine the floating-point grid is right around 1, and it is the single most quoted number describing a format's precision.
Formally, machine epsilon is the gap between 1.0 and the next representable number above it. With a binary significand of p bits (counting the implicit leading 1), that gap is exactly 2^(1-p). In double precision p = 53, so machine epsilon = 2^(-52) ~ 2.22 * 10^(-16); in single precision p = 24, so it is 2^(-23) ~ 1.19 * 10^(-7). Be careful: some references define eps as half this (2^(-p), the unit roundoff) — the two conventions differ by a factor of 2, so always check which one a library means.
Machine epsilon is your yardstick for everything error-related. The number of reliable significant digits is about -log10(eps), so ~16 in double and ~7 in single. When you compare two floats for 'equality' you should test |a - b| <= tol with a tolerance built from eps scaled by the magnitudes involved, never ==. And eps tells you the floor on relative error: no algorithm, however clever, can deliver a relative accuracy finer than about eps from inputs that are themselves only good to eps.
In double precision, 1.0 + 1e-16 evaluates back to exactly 1.0 because 1e-16 is below machine epsilon (~2.2e-16) and rounds off; but 1.0 + 1e-15 gives 1.000000000000001, a different number. The smallest 'detectable' addition to 1.0 is on the order of 2^(-52).
Eps = the spacing of the grid right next to 1.0.
Machine epsilon describes the spacing near 1.0 only; the absolute gap between neighbours is about eps times the number's magnitude, so near a million the steps are millions of times bigger. Relative spacing is what is roughly constant.