semantic segmentation
/ sih-MAN-tik seg-men-TAY-shun /
Semantic segmentation classifies every single pixel in an image. Instead of one label for the whole picture, or boxes around objects, it produces a colored map where each pixel is tagged with what it belongs to: this pixel is road, that one is sky, those are tree, these are person. The result looks like someone carefully painted over the photo by category, leaving no pixel unassigned. It is the most fine-grained way to answer "what is where" in an image.
Compared with a bounding box, this is a leap in precision. A box around a winding road would include grass, sky, and cars; segmentation traces the road's exact, irregular outline. That precision is why it powers self-driving cars (where exactly is the drivable lane?), medical imaging (which pixels are tumor versus healthy tissue?), satellite analysis (forest versus farmland), and photo background removal. The model is trained on images where humans painstakingly labeled every pixel — slow, expensive work that is the field's main bottleneck.
The crucial limitation is in the word "semantic." It labels what each pixel is, but not which instance. If three people stand shoulder to shoulder, semantic segmentation paints all of them the same "person" color in one connected blob — it cannot tell you there are three, or where one ends and the next begins. Separating individuals is a different, harder task called instance segmentation. So semantic segmentation is wonderful for stuff (sky, grass, road) and weaker for counting distinct things.
Feed a street scene to a segmentation model and you get back a recolored image: gray for road, blue for sky, green for trees, red for all pedestrians. The three people waiting at the corner are one continuous red region — the map knows they are "person" but not that they are three separate people.
Every pixel gets a category — but identical-category neighbors merge into one blob.
Semantic segmentation answers "what category is this pixel?" not "which object?" It cannot count or separate touching things of the same class. If you need to tell individual objects apart, that is instance segmentation — a distinct and harder task.