Adversarial Weight Perturbation

AWP (Adversarial Weight Perturbation) is a robust training technique that perturbs both the input AND the model weights during adversarial training — the weight perturbation flattens the loss landscape, leading to smoother minima that generalize better to unseen adversarial examples.

How AWP Works

• Standard AT: Only perturbs inputs — finds worst-case input perturbation $\delta$.
• AWP: Additionally perturbs weights $\theta$ — finds worst-case weight perturbation $\gamma$.
• Double Max: $\min_\theta \max_\gamma \max_\delta L(f_{\theta+\gamma}(x+\delta), y)$ — perturb both weights and inputs.
• Flat Minima: Weight perturbation drives the model toward flat loss landscapes, improving adversarial generalization.

Why It Matters

• Robust Overfitting: Standard adversarial training suffers from robust overfitting — AWP mitigates this.
• State-of-Art: AWP consistently improves adversarial accuracy on top of AT, TRADES, or MART.
• Plug-In: AWP can be added to any adversarial training method as a simple augmentation.

AWP is shaking the model AND the input — double perturbation drives the model to flat, robust loss landscapes that resist adversarial overfitting.