U-Net
/ YOO net /
U-Net is an encoder-decoder shaped like the letter U, designed for tasks where the output is itself a picture the same size as the input — most famously segmentation, where you must label every single pixel ("this pixel is tumor, that one is healthy tissue"). The left arm of the U is an encoder that repeatedly shrinks the image, learning what is in it; the right arm is a decoder that gradually expands back to full resolution, learning where each thing is. The two arms together must answer both "what" and exactly "where."
There is a tension here. As the encoder shrinks the image to grasp the big picture, it throws away fine spatial detail — but pixel-precise output is exactly what segmentation needs. U-Net's signature trick solves this with skip connections: at each level, it copies the high-resolution feature maps straight across from the left arm to the matching spot on the right arm. So the decoder, while rebuilding, gets handed back the crisp edge information the encoder had discarded. Those horizontal copies are the rungs of the U, and they are why borders come out sharp instead of mushy.
U-Net was introduced in 2015 for biomedical images, where labeled examples are scarce, and it works impressively well even with modest training data. It became the standard architecture for medical and scientific image segmentation, and — in a twist few foresaw — the U-Net shape is the backbone inside modern image-generating diffusion models, which repeatedly denoise a picture using exactly this encoder-decoder-with-skips structure. The honest note: U-Net is specialized for grid-like, same-size-out image tasks; it is not a general-purpose network for classification or language.
Hand U-Net a microscope image of cells; it returns a same-size mask coloring every pixel as "cell" or "background." The skip connections carry the cells' crisp outlines from the encoder across to the decoder, so the boundaries come out sharp rather than blurry.
Shrink to understand, expand to localize — skip connections keep the edges crisp.
The skip connections are what make U-Net special, not the U shape itself: they ferry fine spatial detail past the bottleneck so the output's boundaries stay precise. The same structure now powers the denoising step inside image-generating diffusion models.