stack smashing

If a stack buffer overflow is the spill, stack smashing is the deliberate weaponization of that spill: arranging the overflowing bytes so that they land exactly on the function's return address and replace it with an address of the attacker's choosing. The phrase comes from a famous 1996 article, Smashing the Stack for Fun and Profit, which laid out the whole technique step by step.

Walk through the mechanism. A function's stack frame, on a typical downward-growing stack, holds (from lower to higher addresses) its local buffers, then the saved frame pointer (rbp), then the saved return address — the value the ret instruction will pop into rip to resume the caller. An overflow that writes past a local buffer marches upward through memory toward those saved values. The attacker pads the input so the bytes that overwrite the return address form a pointer to code they want to run: in the original attack, to shellcode they also placed in the buffer; in modern attacks (because of DEP/NX), to the start of a return-oriented programming chain. When the function executes ret, it jumps to the attacker's address instead of back to the legitimate caller.

It matters because 'control the return address' is the canonical path from a memory bug to full control-flow hijacking. The honest caveat: nearly every modern compiler and OS now defends this exact spot. A stack canary sits between the locals and the return address and is checked before ret; ASLR randomizes addresses so the attacker does not know where to point; and DEP makes the buffer non-executable. So textbook stack smashing rarely works unmodified today — but the bug class is alive, and chaining it past those mitigations is the real craft.