Hard Example Mining

Hard Example Mining is a training strategy that focuses the model's learning on the most difficult (highest-loss) examples — instead of treating all training samples equally, hard mining identifies and over-represents the challenging examples that drive the most learning.

Hard Mining Methods

• Offline: After each epoch, rank all examples by loss and create a new training set biased toward high-loss examples.
• Online: Within each mini-batch, compute loss on all samples but backpropagate only the top-K hardest.
• Semi-Hard: Focus on examples that are hard but not too hard — avoid outliers and mislabeled data.
• Triplet Mining: For metric learning, mine the hardest positive/negative pairs.

Why It Matters

• Efficiency: Easy examples contribute little to gradient updates — hard mining focuses compute where it matters.
• Imbalanced Data: In defect detection (rare events), hard mining ensures the model focuses on the rare, important cases.
• Convergence: Hard mining accelerates convergence by prioritizing informative gradient updates.

Hard Example Mining is learning from mistakes — focusing training effort on the examples the model finds most challenging.