load balancing

Picture several supermarket checkout lanes: if one cashier has a huge queue while two stand idle, a manager waves shoppers over to spread the crowd. Load balancing in a multicore system is that manager — it works to keep the ready processes spread evenly across the cores, so no core is swamped while others sit doing nothing and overall work gets done faster.

How it works, with per-core run queues: there are two complementary mechanisms. In push migration, a periodic check examines the load on each core and, finding an imbalance, actively pushes processes off an overloaded core onto under-loaded ones. In pull migration, an idle core that has emptied its own queue reaches out and pulls a waiting process from a busier core's queue. Real systems often use both — pull keeps idle cores from wasting cycles, push corrects slower-building imbalances. The trigger is usually a load threshold so the system does not bounce processes around for trivial differences.

Why it matters and its cost: without load balancing, per-core queues can drift badly out of balance, defeating the whole point of having many cores. But balancing is not free, and here is the honest tension: every migration violates processor affinity, forcing a process to start cold on a new core's cache. So aggressive balancing can hurt performance through cache misses, while timid balancing wastes idle cores. Good schedulers balance only when the imbalance is large enough to outweigh the cache penalty — and on NUMA hardware they also try to keep a process near its memory, not just near work.