Class-Balanced Loss

Class-Balanced Loss is a loss function modification that re-weights the loss for each class based on the effective number of samples — addressing class imbalance by assigning higher weight to under-represented classes, preventing the model from being dominated by majority classes.

Class-Balanced Loss Formulation

• Effective Number: $E_n = \frac{1 - \beta^n}{1 - \beta}$ where $n$ is the number of samples and $\beta \in [0,1)$ is the overlap parameter.
• Weight: $w_c = \frac{1}{E_{n_c}}$ — inversely proportional to the effective number of samples in class $c$.
• Loss: $L_{CB} = \frac{1}{E_{n_c}} L(x, y)$ — applies the weight to the standard loss (cross-entropy, focal loss, etc.).
• $\beta$ Parameter: $\beta = 0$ gives uniform weights; $\beta \rightarrow 1$ gives inverse-frequency weights.

Why It Matters

• Long-Tail: Many real-world datasets follow a long-tail distribution — few dominant classes, many rare classes.
• Semiconductor: Defect types follow a long-tail distribution — common defects dominate rare but critical ones.
• Effective Number: Accounts for data overlap — more sophisticated than simple inverse-frequency weighting.

Class-Balanced Loss is weighing by rarity — giving more importance to under-represented classes based on their effective sample count.