u-net

U-Net is the encoder-decoder design so clean and so effective that it became the default for biomedical image segmentation and far beyond. Drawn out, its shape is a literal letter U: the contracting (downsampling) encoder runs down the left arm, the expanding (upsampling) decoder runs up the right arm, and horizontal skip connections bridge across at every level. Introduced by Ronneberger, Fischer, and Brox in 2015 for cell and tissue images, it remains a first-choice baseline a decade later, including as the denoising backbone inside many image diffusion models.

Its defining feature is the skip connection done by concatenation. At each resolution, the high-detail feature maps from the encoder are concatenated onto the upsampled feature maps in the decoder before further processing. This hands the decoder both the abstract "what" from deep in the network and the precise "where" from the matching shallow layer, which is exactly what produces U-Net's famously sharp boundaries even on thin, delicate structures like cell membranes or blood vessels.

Several design choices reflect its medical origins, where labeled data is scarce. The original used unpadded ("valid") convolutions, so the output is smaller than the input, handled by an overlap-tile strategy that mirrors the image at borders to segment large slides seamlessly. It leaned heavily on aggressive data augmentation, especially elastic deformations, to learn from only a handful of annotated images. And it used a pixel-weighted loss that puts extra penalty on the thin gaps between touching cells, forcing the network to learn separating borders. The nnU-Net framework later showed that a well-configured plain U-Net, with automated preprocessing and training recipes, still beats most fancier architectures on medical benchmarks.