regression testing

You spend a week tuning your code, finally get the right answer on a hard problem, and move on. Two months later you 'improve' an unrelated part of the program — and without noticing, you break that hard problem again. Regression testing is the safety net that catches exactly this: a collection of automated tests that re-run after every change to confirm the code still produces the results it used to, so that a 'regression' (something that used to work and now does not) is caught immediately rather than discovered painfully much later.

Mechanically, a regression test pins down a known-good output and checks the current code against it. You run the code on a fixed input, save the answer you trust (sometimes called a golden master or a baseline), and the test re-runs that input and compares. For numerical code there is a crucial subtlety: you almost never test for EXACT equality of floating-point numbers, because round-off, a different BLAS, or a reordered sum can change the last digits legitimately. Instead you assert that the new result is within a stated tolerance of the baseline — for example, that the relative error is below 1e-10. Tests are usually wired into continuous integration, so that every commit automatically triggers the whole suite and a failure is flagged before the change is merged. This is distinct from a unit test (which checks one small function against a known answer) and from verification (which checks correctness against the true math): regression tests guard against CHANGE, ensuring today's code agrees with yesterday's trusted code.

Regression testing is one of the quiet pillars of trustworthy numerical software, and it pairs naturally with reproducibility and benchmarks: a frozen benchmark answer makes an ideal regression baseline. The honest cautions are about the tolerance. Set it too tight and the suite cries wolf on every harmless floating-point wiggle, training the team to ignore failures; set it too loose and a real bug that degrades accuracy slips through unnoticed. Choosing the tolerance is a judgement call that should reflect the problem's actual accuracy needs. And a subtle trap: a regression test only certifies that the answer has not CHANGED — if your baseline was wrong to begin with, the suite will faithfully protect the wrong answer forever. Regression tests preserve behaviour; they do not establish correctness, which is verification's job.