Trades

TRADES (TRadeoff-inspired Adversarial DEfense via Surrogate-loss minimization) is a robust training method that explicitly balances clean accuracy and adversarial robustness — decomposing the robust risk into natural error plus a boundary error regularization term.

TRADES Formulation

• Objective: $\min_\theta \mathbb{E}[underbrace{L(f(x), y)}_{\text{natural loss}} + \beta underbrace{\max_{|\delta|leqepsilon} KL(f(x) | f(x+\delta))}_{\text{robustness regularizer}}]$.
• Natural Loss: Standard cross-entropy on clean inputs (maintains clean accuracy).
• Robustness Term: KL divergence between clean and adversarial predictions (encourages consistent predictions).
• Trade-Off ($\beta$): Higher $\beta$ = more robust but lower clean accuracy. Lower $\beta$ = higher clean accuracy but less robust.

Why It Matters

• Better Trade-Off: TRADES achieves better accuracy-robustness trade-offs than standard adversarial training.
• Theoretical Foundation: Grounded in the decomposition of robust risk (Zhang et al., 2019).
• Tunable: The $\beta$ parameter gives explicit control over the accuracy-robustness trade-off.

TRADES is the balanced defense — explicitly optimizing both clean accuracy and adversarial robustness with a tunable trade-off parameter.