the cost of threads

Threads feel weightless when you write the code - one call to pthread_create() and you have another worker - but they are not free, and treating them as free leads to programs that run worse the more threads you add. There are two main costs to keep in mind: the cost of creating (and destroying) a thread, and the cost of switching between threads while they run. Picture hiring temporary workers: each new hire takes time to onboard and a desk to sit at, and if you have far more workers than desks, they spend the day swapping seats instead of working.

Take the costs in turn. Creating a thread is cheaper than creating a whole process - threads share the address space, so there is less to set up - but it is still real work: the system allocates a fresh stack (often a megabyte or more of address space reserved), sets up bookkeeping, and registers the thread with the scheduler. Spawn thousands of short-lived threads and that setup cost dominates. The second cost is the context switch: every time the OS pauses one thread and resumes another it must save and restore register state, and the switch tends to flush useful data out of the CPU's cache, so the resumed thread runs slowly until its working data is reloaded. When you have far more threads than cores, the cores spend a growing share of their time switching rather than computing.

Why this matters in practice: more threads is not automatically more speed. Past the point where you have roughly as many busy threads as cores, adding more usually makes things slower, because the extra threads add switching and memory-contention overhead without adding any real parallelism (there are no more cores to run them on). The standard professional responses are to reuse a fixed pool of threads instead of creating one per tiny task (a thread pool), and to match the number of active threads to the number of cores. Threads are a powerful tool, but like any resource they have a price, and good systems programmers spend them deliberately.