Balanced Sampling

Balanced Sampling is a data loading strategy that constructs mini-batches with equal (or balanced) representation of each class — ensuring every class appears proportionally in each training batch, regardless of the original class distribution in the dataset.

Balanced Sampling Strategies

• Class-Balanced: Sample equal numbers from each class per batch — each batch has $B/C$ samples per class.
• Square-Root Sampling: Sample proportional to $\sqrt{n_c}$ — a compromise between balanced and natural frequency.
• Progressively Balanced: Start with natural frequency, gradually shift to balanced sampling during training.
• Instance-Balanced: Sample all instances equally, ensuring rare instances get represented.

Why It Matters

• Mini-Batch Coverage: With natural sampling, rare classes may not appear in many mini-batches — balanced sampling ensures coverage.
• Gradient Diversity: Balanced batches provide gradient updates from all classes — better optimization landscape.
• Trade-Off: Fully balanced sampling over-represents rare classes — can cause overfitting on minority classes.

Balanced Sampling is equal airtime for all classes — constructing training batches with proportional class representation regardless of dataset imbalance.