adversarial example

An adversarial example is an input that has been deliberately tweaked by a tiny, often invisible amount so that a model confidently gives the wrong answer, even though a human sees no change. The famous illustration: take a photo of a panda that a network classifies correctly, add a faint, carefully computed noise pattern imperceptible to the eye, and the same network now declares it a 'gibbon' with 99% confidence. The image still looks exactly like a panda to you. This is unsettling because it shows the model's notion of 'panda' is not the human one — it relies on fragile patterns a tiny nudge can break.

To make 'tiny' precise, the perturbation is constrained to be small in some norm. Most commonly an L-infinity bound: every pixel may change by at most ε (epsilon), a small number like 8/255 on a 0–255 scale, so no single pixel shifts visibly. Other threat models use L2 (small total energy) or L0 (few pixels changed, as in patch or sticker attacks). Within that budget, the attacker searches for the perturbation that maximizes the model's loss — formally, find δ with ‖δ‖ ≤ ε that makes the classifier most wrong on input x + δ. The constraint is what makes the change imperceptible; the optimization is what makes it effective.

Why are models so fragile? The leading explanation (Goodfellow et al., 2015) is that deep networks behave too linearly in high-dimensional input space: many small per-pixel changes, each individually negligible, add up coherently along the gradient to a large change in the output logits. A second, complementary view (Ilyas et al., 2019) is that models latch onto 'non-robust features' — genuinely predictive but human-imperceptible statistical cues in the data — so adversarial examples are not bugs but the model exploiting brittle signals that generalize on clean data yet flip under perturbation.

Adversarial examples are not a lab curiosity. They transfer across models (an attack crafted on one network often fools another, enabling black-box attacks without access to the target), they survive printing and recapture (adversarial stop signs and stickers can fool real cameras), and they extend beyond classification to detection, segmentation, and even large vision-language models. They matter for security (face recognition, autonomous driving, content moderation, malware/vision filters) and as a scientific probe: their existence reveals that high test accuracy does not imply human-aligned understanding, motivating the whole field of adversarial robustness.