Swav

SwAV (Swapping Assignments between Views) is a self-supervised learning method that combines contrastive learning with online clustering — assigning augmented views to prototype vectors (cluster centers) and training the network to predict the assignment of one view from the representation of another.

How Does SwAV Work?

• Prototypes: Learnable cluster center vectors ${c_1, ..., c_K}$.
• Process: Encode two views -> compute soft assignments (codes) to prototypes via Sinkhorn-Knopp -> train each view to predict the other view's assignment.
• Swapping: The "swap" predicts view B's cluster assignment from view A's features, and vice versa.
• Multi-Crop: Uses multiple small crops in addition to two standard crops for efficiency.

Why It Matters

• Scalable: No need for large negative sample pools (prototypes are compact representations of the dataset).
• Multi-Crop: The multi-crop strategy provides a significant accuracy boost at minimal compute cost.
• Performance: Competitive with BYOL and SimCLR on ImageNet benchmarks.

SwAV is learning by cluster matching — using the structure of the dataset's natural clusters to guide representation learning.