data parallelism versus task parallelism
Two ways to put many cooks to work. Way one: give every cook the same job — 'each of you peel this pile of potatoes' — and split one big pile of potatoes among them. Way two: give each cook a different job — one chops onions, one stirs the sauce, one bakes the bread — all at once. The first is data parallelism: the same operation applied to many pieces of data in parallel. The second is task parallelism: different operations running in parallel, each doing its own distinct thing.
Data parallelism splits the data and runs identical work on each chunk simultaneously: add 1.0 to every element of a million-element array by handing each core a slice, or sum a huge dataset by having each worker total a portion and then combining (a map-reduce). The work is uniform; what differs is which slice of data each worker holds. This is the natural fit for parallel-for, GPUs (thousands of cores doing the same instruction on different data), and SIMD instructions on a CPU. Task parallelism instead splits the work into different activities that run concurrently: in a web request, fetch the user profile, query the recommendations service, and load the ad — three unrelated tasks at once, each different code. The work is heterogeneous; what they share (if anything) is only their results being combined at the end. Fork-join can express either, but task parallelism is where the sub-tasks do genuinely different things, while data parallelism is many copies of one thing.
The distinction matters because it points you at the right tools and the right limits. Data parallelism scales beautifully with more cores when the per-element work is independent — it is what GPUs and SIMD are built for, and it is usually easier to reason about because every worker runs the same code. Task parallelism is bounded by how many genuinely independent activities you have and how they depend on each other (their dependency graph), and by Amdahl's law on the serial glue between them. Honest caveat: the two are not exclusive — real systems mix them (a task-parallel pipeline whose stages are each data-parallel). And neither removes the need for synchronisation when tasks touch shared mutable state; 'parallel' describes the shape of the work, not a guarantee of safety.
Data-parallel (one op, many data): pixels.par_iter_mut().for_each(|p| *p = brighten(*p)); — every worker runs brighten on its slice. Task-parallel (many ops): join(fetch_profile(), fetch_ads(), fetch_recs()); — three different jobs at once.
Data parallelism = same code over different data; task parallelism = different code running concurrently.
The two are not mutually exclusive — real workloads combine them. And 'parallel' is about the shape of the work, not safety: when parallel workers share mutable state, you still need synchronisation, whichever kind of parallelism it is.