Lipschitz Constant Estimation

Lipschitz Constant Estimation is the computation or bounding of a neural network's Lipschitz constant — the maximum ratio of output change to input change, $|f(x_1) - f(x_2)| \leq L |x_1 - x_2|$, measuring the network's maximum sensitivity to input perturbations.

Estimation Methods

• Naive Bound: Product of weight matrix operator norms across layers — fast but often very loose.
• SDP Relaxation: Semidefinite programming relaxation for tighter bounds (LipSDP).
• Sampling-Based: Estimate a lower bound by sampling many input pairs and computing maximum slope.
• Layer-Peeling: Tighter compositional bounds that exploit network structure.

Why It Matters

• Robustness Certificate: $L$ directly gives the maximum prediction change for any $epsilon$-perturbation: $Delta f leq L epsilon$.
• Sensitivity: Small Lipschitz constant = stable, robust model. Large = potentially sensitive and fragile.
• Regularization: Training to minimize $L$ (Lipschitz regularization) directly improves adversarial robustness.

Lipschitz Estimation is measuring maximum sensitivity — bounding how much the network's output can change for a given input perturbation.