Pseudo-Labeling with Confidence

Pseudo-Labeling with Confidence is a semi-supervised learning technique that uses the model's own high-confidence predictions on unlabeled data as training labels — filtering predictions by a confidence threshold to ensure only reliable pseudo-labels are used.

How Does It Work?

• Predict: Run unlabeled data through the current model.
• Filter: Keep only predictions where $max(p(y|x)) > \tau$ (confidence threshold, typically $\tau = 0.95$).
• Train: Use filtered pseudo-labeled data alongside labeled data with cross-entropy loss.
• Iterate: Retrain or update the model, then re-predict and re-filter.

Why It Matters

• Simplicity: The simplest semi-supervised learning method — no architectural changes needed.
• FixMatch: The confidence threshold is the core component of FixMatch and modern semi-supervised methods.
• Self-Training: A form of self-training that bootstraps labeled data from model confidence.

Pseudo-Labeling is the model teaching itself — using high-confidence predictions as targets to leverage the vast pool of unlabeled data.