fast gradient sign method

The Fast Gradient Sign Method is the simplest and most influential recipe for manufacturing an adversarial example. The intuition: training a network uses gradients to nudge the weights so the loss goes down; FGSM flips this idea and uses the gradient with respect to the input image to nudge the pixels so the loss goes up — making the model more wrong. It does this in a single step, which is why it is 'fast': one forward pass and one backward pass produce the attack, no iterative search.

Concretely, the perturbation is δ = ε · sign(∇ₓ J(θ, x, y)). Reading this in plain words: ∇ₓ J is the gradient of the loss J with respect to the input pixels x (holding the trained weights θ and true label y fixed) — it points in the direction in pixel space that increases the loss fastest. The sign(·) function reduces every component to just its direction, +1 or −1, discarding magnitude. Multiplying by ε scales every pixel's change to exactly ±ε. The adversarial image is x_adv = x + δ. Using the sign (rather than the raw gradient) is precisely what makes the change fit an L-infinity budget: every pixel moves by the same small amount ε, so the maximum per-pixel change is bounded and the result stays imperceptible.

Why does one step work at all? FGSM rests on the linear-explanation of adversarial examples: in high dimensions, the loss is locally well-approximated by a linear function, and the change in loss from moving each pixel by ±ε in the loss-increasing direction is ε times the sum of absolute gradient components — which grows with the number of pixels. So even though ε is tiny per pixel, the millions of pixels combine into a large, coherent push on the output. The sign step aligns every pixel's contribution to add up rather than cancel.

FGSM is the foundation on which stronger attacks are built. Iterating it in small steps with re-clipping to the ε-ball gives BIM/I-FGSM and the standard benchmark PGD (Projected Gradient Descent), which usually fools models FGSM cannot; adding momentum (MI-FGSM) improves transferability across models. FGSM itself is a white-box attack (it needs the model's gradients), and it is mainly used today as a quick robustness check and, crucially, as the engine of adversarial training — generating adversarial examples on the fly to train against. A known pitfall is that training only against single-step FGSM can cause 'gradient masking', where the model learns to obscure its gradients and looks robust to FGSM while remaining vulnerable to multi-step PGD.