Yopo

YOPO (You Only Propagate Once) is a fast adversarial training method based on the observation that adversarial perturbations mainly depend on the first layer's gradients — by restricting full backpropagation to the first layer and updating the perturbation with cheap first-layer gradient computations.

How YOPO Works

• Key Insight: The adversarial perturbation $\delta$ is an input-space quantity — its gradient primarily depends on the first layer.
• Full Backprop: Perform one full forward-backward pass to update model weights.
• Cheap Updates: Perform $p$ additional cheap perturbation updates using only the first layer's gradient.
• Cost Reduction: Full backprop once + $p$ cheap first-layer passes ≈ $1 + p \cdot \epsilon$ forward-backward cost (where $\epsilon ll 1$).

Why It Matters

• Theoretical Foundation: Based on the Pontryagin's Maximum Principle (PMP) connection to adversarial training.
• Efficiency: Achieves PGD-level robustness with significantly fewer full backward passes.
• Scalable: The first-layer gradient computation is much cheaper than full backpropagation.

YOPO is cheap perturbation updates — exploiting the structure of adversarial perturbations to avoid repeated full backpropagation.