interactive segmentation

Interactive segmentation keeps a human in the loop: rather than predicting a mask fully automatically, the system takes a few cheap hints from a person and produces (then refines) the mask around them. The hints are deliberately minimal — a click inside the object, a click on the background to push the mask back, a scribble, or a bounding box. The goal is to reach a high-quality mask in as few interactions as possible, which is exactly what you want when annotating training data or editing a photo where full automation would be too unreliable.

The classical tools were energy-minimization methods: GrabCut (rectangle plus optional brush strokes), Intelligent Scissors / live-wire (the cursor snaps to edges as you trace), and the random walker (label propagation from seed pixels). Modern deep approaches feed the user's clicks to the network as extra input channels (often encoded as distance maps from positive and negative clicks) and predict an updated mask each time; representative systems include DEXTR (segment from four extreme points), f-BRS, RITM, and transformer-based SimpleClick. SAM can be seen as a foundation-model take on the same interface — a click yields an instant mask.

The standard way to measure these methods reflects their purpose: report the number of clicks needed to reach a target overlap, for example "clicks to 90% IoU" (NoC@90), averaged over a dataset. A good interactive model also has to behave sensibly — each correction click should reliably improve the mask, not thrash it — which is why click simulation during training (placing the next click at the worst current error) is a core technique. Interactive segmentation is the engine behind most large mask-annotation efforts, since a person verifying and nudging a model is far cheaper than drawing every boundary by hand.