instance segmentation
/ IN-stuns seg-men-TAY-shun /
Instance segmentation is the most demanding everyday vision task: for each separate object, it produces a pixel-perfect mask — the exact silhouette — and tells the objects apart even when they belong to the same category. Picture three overlapping sheep: instance segmentation outputs three distinct masks, one per sheep, each tracing that animal's precise outline. It is detection (which objects, where) and segmentation (exact shape) fused, with the added power to count and individuate.
Think of it as the union of two earlier ideas. Object detection gives you a box per object and can count them, but the box is a crude rectangle. Semantic segmentation gives exact pixel outlines, but merges same-category objects into one blob. Instance segmentation delivers both at once: this pixel is person, and specifically person number two. The best-known method, Mask R-CNN, literally bolts a mask-predicting branch onto a Faster R-CNN detector — find each object, then paint its precise shape.
It powers tasks where you must isolate individual objects cleanly: a robot picking one item from a cluttered bin, counting cells in a microscope slide, separating each car for traffic analysis, or cutting a person out of a photo. The honest costs are steep. Training data is brutally expensive — humans must hand-trace every object's outline — and the models are heavier and slower than plain detection. Boundaries between touching, look-alike objects remain the hardest place for errors to creep in.
A warehouse robot looks into a bin of identical boxes. Instance segmentation returns one separate mask per box — box 1, box 2, box 3 — each outlined exactly, so the gripper knows precisely where one box ends and the next begins, even where they overlap.
Same category, but each object gets its own mask — detection plus exact shape plus a count.
Instance segmentation is what people often imagine "the AI sees objects" really means — but it is one of the costliest tasks to label and run. There is also panoptic segmentation, which combines instance masks for countable things with semantic regions for amorphous stuff like sky and road.