pix2pix

pix2pix (Isola et al., 2017) is the paired counterpart to CycleGAN: it learns image-to-image translation when you do have matched before/after pairs — edges to photo, map to aerial image, day to night of the exact same scene, sketch to rendered object. Because each input comes with its correct output, the model can be told exactly what "right" looks like. The insight is to combine two pressures: a reconstruction loss that says "match the known target", and an adversarial loss that says "and also look like a real image", which together avoid the blurriness a reconstruction loss alone would cause.

pix2pix is a conditional GAN conditioned on the input image. The generator G(x) maps input image x to an output, trained with two terms: the conditional adversarial loss (a discriminator D(x, y) judges whether the pair (input, output) is a real matched pair) plus an L1 reconstruction loss, λ times the L1 norm of y − G(x), to the ground-truth target. L1 (rather than L2) is used because it blurs less; the adversarial term supplies the high-frequency realism that L1 alone misses. The generator is a U-Net (an encoder–decoder with skip connections so low-level detail and structure bypass the bottleneck), and the discriminator is a PatchGAN that classifies each N×N image patch as real or fake and averages — focusing the adversarial signal on local texture while L1 handles global low-frequency correctness.

The clean idea is a division of labor: L1 gets the overall structure and colors approximately right (the low frequencies), and the PatchGAN adversarial loss makes the textures crisp and plausible (the high frequencies). pix2pix is a general-purpose framework — the same architecture handles many tasks just by changing the training pairs — and it set the template later extended by pix2pixHD (high-resolution, multi-scale) and SPADE / GauGAN (semantic-map to photo). Its hard requirement is aligned paired data; when pairs are unavailable, you drop to unpaired methods like CycleGAN, trading reliability for not needing pairs.