Mixup

Mixup is a data augmentation technique that creates new training samples by linearly interpolating between pairs of existing samples and their labels — encouraging the model to learn smooth, linear decision boundaries between classes.

How Does Mixup Work?

• Sample: Draw mixing coefficient $\lambda \sim \text{Beta}(\alpha, \alpha)$ (typically $\alpha = 0.2$).
• Mix Inputs: $\tilde{x} = \lambda x_i + (1-\lambda) x_j$.
• Mix Labels: $\tilde{y} = \lambda y_i + (1-\lambda) y_j$.
• Train: Use $(\tilde{x}, \tilde{y})$ as a regular training sample with cross-entropy loss.
• Paper: Zhang et al. (2018).

Why It Matters

• Smoother Boundaries: Linear interpolation encourages linear behavior between classes -> better calibration.
• Regularization: Acts as a strong regularizer, reducing overfitting especially on small datasets.
• Universal: Works for images, text, audio, tabular data — any domain where interpolation is meaningful.

Mixup is blending reality — creating in-between examples that teach the model smooth, calibrated transitions between classes.