Linear Probing

Linear Probing is an evaluation protocol for pre-trained representations where a single linear layer is trained on top of frozen features — used to measure how linearly separable the learned features are, serving as a standardized benchmark for representation quality.

How Does Linear Probing Work?

• Freeze: The entire pre-trained backbone. No gradients flow through it.
• Train: Only a linear classifier (fully connected layer + softmax) on the frozen features.
• Dataset: Typically ImageNet-1k (1.28M labeled images, 1000 classes).
• Metric: Top-1 accuracy. Higher = better representations.

Why It Matters

• Standardized Benchmark: The primary way to compare SSL methods (SimCLR, MoCo, DINO, MAE, etc.).
• Measures Separability: If features are linearly separable, the pre-training learned a meaningful structure.
• Conservation: No fine-tuning means the result strictly measures the pre-trained features, not the model's ability to adapt.

Linear Probing is the straight-line test for representations — measuring whether pre-trained features organize themselves into linearly separable clusters.