High-Performance & Parallel Computing

a race condition

Two people share a bank account with a balance of 100. Both decide to deposit 50 at the same instant. Each reads the balance (100), each adds 50, each writes back 150. The final balance is 150, not the correct 200 — one deposit vanished, because the two operations interleaved in a way nobody intended. That is a race condition: when the result of a computation depends on the unpredictable timing of two or more parallel actions touching the same data, and at least one of them is writing.

In parallel code the same bug appears whenever multiple threads read and update a shared variable without coordination. The trouble is that 'increment a counter' is not one indivisible step on real hardware — it is read the value, add one, write it back, three steps that the system can freely interleave between threads. If thread A reads, then thread B reads the same old value, then both write, one update is lost. The cure is synchronisation: a lock (mutex) that lets only one thread into the critical section at a time, an atomic operation that performs the read-modify-write as one indivisible unit, or a restructuring that gives each thread its own private accumulator and combines them only at the end (the standard pattern for parallel reductions). Each costs something — locks serialise and can throttle scaling; over-synchronising can erase the very parallelism you wanted.

Race conditions are the signature hazard of shared-memory parallelism, and they are vicious precisely because they are nondeterministic: the program may give the right answer a thousand times and the wrong one on the thousand-and-first, depending on timing that shifts with load, core count, or the phase of the moon. They do not show up reliably in testing and can hide for years. This nondeterminism also has a subtler face in numerics: even a correctly synchronised parallel sum can give slightly different results from run to run, because the order in which the partial sums are combined varies and floating-point addition is not associative — so bit-for-bit reproducibility is a real and separate concern from correctness.

Eight threads each run sum += a[i] over their slice of an array, all updating the one shared sum. Without protection, updates collide and the total comes out too small and different each run. The fix: give each thread a private partial sum, and combine the eight partials once at the end (a reduction) — no collisions, correct total, full parallelism.

Unsynchronised shared updates lose data nondeterministically; private partials plus a reduction fix it.

Race conditions are nondeterministic, so passing a test once proves nothing — the bug can appear only under a particular timing, core count, or load. And note that even race-free parallel summation is not bit-reproducible, because the order of floating-point additions varies and that addition is not associative; correctness and reproducibility are two different problems.

Also called
data racerace hazard資料競爭競態條件