swav

SwAV (Swapping Assignments between Views) replaces pairwise comparison with online clustering. Instead of asking are these two embeddings close, it asks do these two views belong to the same cluster, and learns by predicting the cluster assignment of one view from the features of another view. This sidesteps the need to compare every image against every other, so SwAV needs no large bank of explicit negative pairs, yet it still keeps representations from collapsing.

The mechanism uses a fixed set of K trainable prototype vectors, think of them as learned cluster centres. For each view, you compute soft cluster assignments (called codes) by comparing its embedding to the prototypes. The training objective is the swapped prediction: from view A's features you must predict view B's code, and vice versa. Because two augmentations of the same image should fall in the same cluster, this enforces consistency through the cluster layer rather than through direct embedding distances. Assignments are computed online within each batch, so no offline clustering pass over the dataset is needed.

The trick that prevents the obvious collapse, where every image is dumped into one cluster, is the Sinkhorn-Knopp algorithm, which solves an optimal-transport problem to make the cluster assignments within a batch equally balanced across the K prototypes. This equipartition constraint is SwAV's anti-collapse mechanism. SwAV also introduced multi-crop, taking two standard-resolution global crops plus several low-resolution local crops of each image, which gives more views at little extra compute and substantially improves results; multi-crop was so effective it was later adopted by DINO and other methods.