Distribution Alignment

Distribution Alignment is a technique in semi-supervised learning that adjusts pseudo-label distributions to match the true class distribution — preventing the model from being biased toward classes it finds easy to predict and ensuring balanced utilization of pseudo-labels.

How Does Distribution Alignment Work?

• Estimate: Track the running average of pseudo-label class distribution $\hat{p}(y)$.
• Target: The expected class distribution $p(y)$ (uniform for balanced datasets, or estimated from labeled data).
• Align: Adjust predictions: $\tilde{p}(y|x) = p(y|x) \cdot p(y) / \hat{p}(y)$ (reweight to match target distribution).
• Normalize: Renormalize the adjusted distribution to sum to 1.

Why It Matters

• Class Balance: Prevents positive feedback loops where easy classes dominate pseudo-labels.
• Long-Tail: Critical for class-imbalanced datasets where some classes are rarely predicted.
• MixMatch/ReMixMatch: Distribution alignment is a key component of these popular semi-supervised methods.

Distribution Alignment is class balance enforcement for pseudo-labels — correcting the model's class biases to ensure all classes are fairly represented.