Fast Adversarial Training

Fast Adversarial Training is a computationally efficient variant of adversarial training that uses single-step attacks (FGSM) instead of multi-step PGD — reducing the training cost from ~10× standard training (PGD-AT) to ~2× while maintaining competitive robustness.

How Fast AT Works

• FGSM + Random Init: Use FGSM with random initialization instead of multi-step PGD.
• Single Step: Only one gradient computation per adversarial example (vs. 7-20 for PGD).
• Catastrophic Overfitting: Na ̈ive FGSM-AT can suffer from catastrophic overfitting — robustness suddenly drops to 0%.
• Fixes: Random initialization, gradient regularization (GradAlign), and early stopping prevent catastrophic overfitting.

Why It Matters

• Speed: ~5× faster than PGD-AT — makes adversarial training practical for large models.
• Accessibility: Enables adversarial training on limited compute budgets.
• Surprising Effectiveness: With proper initialization, single-step FGSM-AT achieves ~90% of PGD-AT robustness.

Fast AT is adversarial training on a budget — using single-step attacks for efficient robust training with proper safeguards against catastrophic overfitting.