the SIMT model
/ sim-tee /
SIMD asks you to think in lanes: one instruction, eight numbers, packed into one wide register that you manage by hand. SIMT lets you think in threads instead: you write a small program for one element — 'take my element, do this to it' — and the hardware runs thousands of copies of that same little program, one per data element, automatically keeping a big group of them in step. SIMT stands for single instruction, multiple threads: it is the GPU's programming model, and it feels like writing ordinary thread code while the hardware executes it SIMD-style underneath.
Here is how the trick works. The programmer writes one function (a 'kernel') describing what one thread does to one element. The GPU launches an enormous number of threads — say one per pixel or per array element — and bundles them into fixed-size groups (NVIDIA calls a group of 32 a warp; AMD calls 64 a wavefront). All threads in one bundle share a single instruction-fetch unit and march in lockstep: at each step the hardware fetches one instruction and every thread in the bundle executes it on its own data, exactly the SIMD idea, but each thread also has its own registers and its own program counter so it feels independent. You get the easy mental model of threads with the efficiency of one instruction driving many.
The honesty SIMT demands is about what 'in lockstep' costs. Because a whole bundle shares one instruction stream, the threads are only truly parallel when they all want to do the same thing. The moment they disagree — some take the if-branch and others the else-branch — the hardware cannot run both at once on shared fetch hardware, so it runs one side with the other threads idle, then the other side, serializing the two paths. That is branch divergence, the central tax of SIMT. So SIMT gives beginners a comfortable thread-like model, but performance still obeys the SIMD reality underneath: regular, agreeing threads fly; divergent, branchy threads crawl.
A vector-add kernel: each thread computes c[id] = a[id] + b[id] where id is its own index. Launch one thread per array element; the GPU groups them into warps of 32, and within a warp all 32 threads execute the load, add, and store in step on their 32 different elements.
Write one thread's worth of code; the hardware runs thousands of copies, bundling them so one instruction drives a whole warp.
SIMT's thread-like view is a convenience, not a guarantee of independence. Threads in one warp share an instruction stream, so when they branch differently the hardware serializes the paths (divergence). The thread model hides the SIMD machine but cannot escape its rules.