The screen lies a little — and that is fine
In the previous guide we said the whole point of computational mathematics is to solve problems with a numerical algorithm that returns a number. Here is the catch that defines the entire subject: that number is almost always an approximation, not the exact answer. When you compute sqrt(2) the machine hands you 1.4142135623730951 — a fine, useful value, but sqrt(2) is irrational and has no finite decimal, so the true number was rounded the moment it was stored. The honest mindset of this field is not 'is my answer right?' but 'how wrong is it, and can I bound that?'
This is the deep break from the pure or symbolic mathematics you have met before. Symbolic computation can keep sqrt(2) as the exact symbol sqrt(2) forever and never lose a thing — but it pays for that exactness with expression swell, where intermediate formulas balloon out of control. Numerical computation trades that perfection for speed and scale: it accepts a tiny, controlled wrongness in every step so it can handle problems with a billion unknowns. The art is keeping that wrongness small and knowing how big it is.
The four doorways error walks through
Error is not one thing; it enters through four distinct doorways, and it helps enormously to name which one you are fighting. The first is modeling error: before any computing happens, you replace messy reality with an equation. Modeling error is the gap between the real world and the model — ignoring air resistance, assuming a beam is perfectly straight, pretending the population grows exactly exponentially. No algorithm, however clever, can recover what the model threw away.
The second doorway is data error: the numbers you feed in are themselves measured, and measurements have noise. If you know a temperature only to within 0.5 degrees, no computation can pretend the output is good to ten digits. The third is truncation error, the most characteristic of numerical math. To make calculus computable we cut an infinite process short: we stop a series after a few terms, or replace a derivative by a difference over a finite step. That deliberate cut is truncation error. When the infinite thing being cut is a continuous domain chopped into a grid of finite cells, the same idea wears the name discretization error.
The fourth doorway is round-off error, and it is unavoidable on any real machine. A computer stores numbers in floating-point, a finite set of values, so most reals get nudged to the nearest representable neighbour — every single operation can round. Crucially, 0.1 has no exact binary form (it is a repeating fraction in base 2), and floating-point addition is not even associative: (a + b) + c can differ from a + (b + c). Round-off is usually minuscule, on the order of machine epsilon, about 1e-16 in double precision — until catastrophic cancellation subtracts two nearly equal numbers and amplifies it dramatically.
How big? Absolute versus relative error
Once we admit the answer is wrong, we need to measure how wrong. If the true value is x and the computed value is x_hat, the absolute error is |x - x_hat| — the raw gap. But a gap of 1 means very different things for x = 3 versus x = 3 billion. So we usually prefer the relative error, |x - x_hat| / |x|, which is scale-free and reads naturally as a percentage or as a number of correct digits: a relative error near 1e-6 means roughly six good significant figures.
absolute error = |x - x_hat|
relative error = |x - x_hat| / |x| (x != 0)
x = 100.0, x_hat = 100.1
absolute = 0.1 relative = 0.001 (~3 digits)
x = 0.001, x_hat = 0.0011
absolute = 0.0001 relative = 0.1 (~1 digit!)Relative error connects straight to a sober limit of double precision: you start with only about 16 correct digits, and they are a budget you spend, never refill. If a problem is ill-conditioned — meaning its condition number is large, say around 1e8 — then small input wobbles get amplified by that factor, and you lose roughly 8 of your 16 digits before any algorithm even runs. This is the preview of the rule we will meet next: accuracy depends on the problem's conditioning as much as on your method, and we will save it for guide 3 and the conditioning rung.
Big-O, two completely different ways
Here is a point that quietly confuses every newcomer: the same big-O notation is used for two entirely different things in this subject, and you must always ask which. The first is order of accuracy — how fast the error shrinks as you refine. A finite-difference scheme might have error O(h^p) in its step size h: a first-order method (p = 1) halves its error when you halve h, while a second-order method (p = 2) cuts the error by four. Bigger p is dramatically better, because errors fall like a power of a quantity you are driving toward zero.
The second use is cost — how the work grows as the problem gets bigger. This is the same computational complexity you would meet in any algorithms course: solving a dense linear system A x = b by elimination costs about O(n^3) operations for an n-by-n matrix, while the fast Fourier transform turns an O(n^2) computation into a celebrated O(N log N). Notice the contrast in direction. For accuracy you want a small quantity (h) sent toward zero; for cost you want a large quantity (n) sent toward infinity. They even point opposite ways: a method that is more accurate per step often costs more per step.
Convergence: chasing the answer, step by step
Many numerical methods do not produce the answer in one shot; they produce a sequence of guesses x_0, x_1, x_2, ... that, you hope, close in on the true x. The method converges if the error e_n = |x - x_n| goes to zero. But two converging methods can race at wildly different speeds, and the order of convergence tells you which. Roughly, if e_{n+1} is proportional to (e_n)^q, then q is the order: q = 1 is linear convergence (the error shrinks by a steady factor each step), while q = 2 is quadratic — the number of correct digits roughly doubles every step.
The classic example is Newton's method for solving f(x) = 0, which we will study in depth later. Its update is x_{n+1} = x_n - f(x_n)/f'(x_n), and near a simple root it is quadratic: spectacularly fast. But be honest about the fine print — that quadratic speed only holds near a simple root with a good starting guess. With a bad start, or a flat spot where f'(x) is near zero, Newton's method can overshoot wildly, cycle forever, or diverge. Fast convergence is a local promise, not a guarantee from anywhere.
Finally, a misconception worth killing early: with truncation error like O(h^p), it is tempting to think 'just take h as small as I can and the error vanishes.' On a real machine it does not. As h shrinks, truncation error falls — but round-off error from subtracting nearly equal numbers grows, so the total error bottoms out and then gets worse. There is an optimal h, a step-size tradeoff, beyond which smaller steps hurt. Every source of error from this guide is alive at once, and computing well means balancing them rather than chasing any single one to zero.