a benchmark problem

Before you trust a new car, you might drive it around a familiar test track where you already know how fast the laps should be and where the tricky corners are. A benchmark problem is the test track for a piece of numerical software: a standard, well-documented problem with a known or widely-agreed answer, that the whole community uses to check and compare codes. When your brand-new solver gets the benchmark right, you gain confidence it will get unfamiliar problems right; when codes disagree on a benchmark, the community learns something.

Benchmarks come in a few flavours. Some have an exact closed-form answer (a manufactured solution, or a problem solvable by hand), and these are used for verification — your code's error against the exact answer must shrink at the right rate. Others have no closed form but a 'reference' answer agreed by many high-quality independent computations or by careful experiment; these are used both to validate models and to compare different methods fairly. Classic examples in different fields include the lid-driven cavity flow in computational fluid dynamics (a square box with one moving wall, whose flow pattern is tabulated to many digits), standard test matrices for linear algebra, and stiff ODE test sets. A good benchmark comes with a precise specification — exact geometry, boundary conditions, parameters — so that two groups solving 'the same' problem really are.

Benchmarks matter because they make trust transferable and comparison fair. They let a reviewer reproduce your claim, let a newcomer calibrate a code against established results, and let competing methods be judged on identical ground. They are also a key ingredient of regression testing: freeze a benchmark's answer and re-run it after every code change to catch accidental breakage. The honest caveats: passing a benchmark proves the code handles THAT problem, not all problems — a code can be tuned (even unconsciously) to ace a famous benchmark while failing on the case you actually care about. And a benchmark's 'reference' answer is only as good as the computations or experiments behind it; reference values have, on occasion, been revised. A benchmark is necessary evidence of correctness, never sufficient proof.