Welsch Loss

Welsch Loss is a robust loss function that bounds the maximum penalty for outliers — using an exponential form $L(r) = \frac{c^2}{2}[1 - \exp(-(r/c)^2)]$ that asymptotes to a constant for large residuals, preventing outliers from dominating the optimization.

Welsch Loss Properties

• Form: $L(r) = \frac{c^2}{2}[1 - \exp(-r^2/c^2)]$ — converges to $c^2/2$ as $|r| \rightarrow \infty$.
• Small Residuals: Behaves like squared loss for $|r| ll c$ — standard quadratic behavior.
• Large Residuals: Loss saturates at $c^2/2$ — outliers have bounded, constant influence.
• Parameter $c$: Controls the transition between quadratic and constant regions (inlier-outlier threshold).

Why It Matters

• Robust Regression: Completely eliminates the influence of extreme outliers — they can't dominate the loss.
• Process Data: Semiconductor process data often contains outliers from sensor failures — Welsch loss prevents corruption.
• Smooth: Unlike Huber loss (which has a slope change at the threshold), Welsch loss is infinitely smooth.

Welsch Loss is the gentlest robust loss — smoothly transitioning from quadratic to bounded behavior for complete outlier immunity.