Wasserstein GAN (WGAN)

Classic GANs train a discriminator to classify real versus fake, but when the generated and real distributions barely overlap, that classifier saturates and hands back nearly zero gradient, leaving the generator stuck. Wasserstein GAN swaps the classification objective for a smoother yardstick of distance: the Earth-Mover distance, which measures the minimum cost of transporting one distribution's mass to match the other. Because this distance varies smoothly even between non-overlapping distributions, it supplies useful gradients everywhere and largely tames training instability and mode collapse.

The construction uses the Kantorovich-Rubinstein duality, which rewrites the Earth-Mover distance as a supremum over all one-Lipschitz functions of the difference in their expectations under the two distributions. The discriminator, now called a critic, outputs an unbounded real score and is trained to maximize that difference, so its value approximates the Wasserstein distance and its gradient guides the generator. The catch is enforcing the one-Lipschitz constraint; the original paper clipped weights, but the gradient-penalty variant, which penalizes the critic's gradient norm away from one, is far more reliable.

WGAN reframed adversarial training as estimating an optimal-transport distance, gave a loss that correlates with sample quality, and made GAN training markedly more stable, influencing much of the GAN literature that followed.