Robust Training Methods

Robust Training Methods are training algorithms that produce neural networks resilient to adversarial perturbations, noise, and distribution shift — going beyond standard ERM (Empirical Risk Minimization) to explicitly optimize for worst-case or perturbed-case performance.

Key Robust Training Approaches

• Adversarial Training (AT): Train on adversarial examples generated during training (PGD-AT).
• TRADES: Trade off clean accuracy and robustness with an explicit regularization term.
• Certified Training: Train to maximize certified robustness radius (IBP training, CROWN-IBP).
• Data Augmentation: Heavy augmentation (AugMax, adversarial augmentation) improves distributional robustness.

Why It Matters

• Standard Training Fails: Standard ERM produces models that are trivially fooled by small perturbations.
• Defense: Robust training is the most effective defense against adversarial attacks — far better than post-hoc defenses.
• Trade-Off: Robust models typically sacrifice some clean accuracy for improved worst-case performance.

Robust Training is training for the worst case — explicitly optimizing models to maintain performance under adversarial and noisy conditions.