Exemplar Learning
Exemplar learning is the early self-supervised approach that groups multiple augmentations of the same image into one pseudo-class to learn invariant features - it predated large-scale contrastive pipelines and demonstrated that transformation consistency can supervise representation learning.
What Is Exemplar Learning?
- Definition: Generate transformed variants of each image and train network to treat those variants as related exemplars.
- Pseudo-Label Strategy: Each source image forms a pseudo category under augmentation.
- Objective Choices: Triplet loss, pairwise metric losses, or proxy classification variants.
- Historical Context: Important stepping stone toward modern instance contrastive methods.
Why Exemplar Learning Matters
- Invariance Learning: Encourages robustness to rotation, crop, color, and geometric transformations.
- Label-Free Supervision: Uses synthetic relationships without manual annotation.
- Method Simplicity: Clear augmentation-driven supervisory signal.
- Legacy Influence: Inspired later methods that formalized positive-pair construction.
- Educational Value: Useful baseline for understanding SSL objective evolution.
How Exemplar Learning Works
Step 1:
- Apply multiple stochastic augmentations to each image to create exemplar set.
- Encode exemplars into embedding space with shared backbone.
Step 2:
- Optimize metric objective so exemplars from same source are close and others remain separated.
- Repeat across dataset to build transformation-invariant representation geometry.
Practical Guidance
- Augmentation Diversity: Too weak gives poor invariance, too strong can remove semantics.
- Triplet Sampling: Hard negative mining often improves convergence quality.
- Scale Limits: Large pseudo-class counts can stress memory and classifier design.
Exemplar learning is an early but influential SSL strategy that proved augmentation consistency can replace manual labels for representation training - it remains a useful conceptual baseline for modern self-supervised pipelines.