Focal Loss

Focal loss is a modified cross-entropy loss function designed to address extreme class imbalance — by down-weighting well-classified (easy) examples and focusing training on hard, misclassified samples, focal loss enables single-stage object detectors like RetinaNet to achieve accuracy comparable to two-stage detectors.

Why Focal Loss Matters

• Class Imbalance: In object detection, background patches outnumber objects 1000:1.
• Easy Example Problem: Standard cross-entropy wastes gradient on trivially classified negatives.
• Hard Mining Alternative: Focal loss automates what hard negative mining does manually.
• Single-Stage Detectors: Made RetinaNet competitive with Faster R-CNN.

Formula

FL(p) = −α(1 − p)^γ log(p), where γ (gamma, typically 2) controls the focusing strength and α balances class weights.

Impact by Example Difficulty

• Easy example (p=0.9): Loss reduced to 0.01% of standard CE.
• Hard example (p=0.1): Loss remains at ~90% of standard CE.
• Result: Model focuses almost entirely on difficult, informative examples.

Applications: YOLO, RetinaNet, SSD, medical image segmentation, fraud detection, any task with severe class imbalance.

Focal loss transformed object detection — proving that class imbalance, not architecture, was the main barrier to single-stage detector performance.