Federated Proximal

FedProx (Federated Proximal) is an improvement to FedAvg that adds a proximal term to the local objective — penalizing local models that drift too far from the global model, improving convergence under heterogeneous (non-IID) data distributions and variable client compute.

FedProx Formulation

• Local Objective: $\min_w L_k(w) + \frac{\mu}{2}|w - w_t|^2$ — local loss + proximal term.
• Proximal Term: $\frac{\mu}{2}|w - w_t|^2$ prevents the local model from drifting too far from the global model.
• $\mu$ Parameter: Controls the penalty strength — larger $\mu$ = stronger pull toward global model.
• Partial Work: FedProx handles variable compute — clients can perform different numbers of local steps.

Why It Matters

• Non-IID Data: FedAvg diverges with highly non-IID data — FedProx stabilizes convergence.
• System Heterogeneity: Different clients may have different compute capabilities — FedProx handles partial work.
• Simple Fix: Just one additional term to the local loss — drop-in replacement for FedAvg.

FedProx is FedAvg with a leash — keeping local models from straying too far from the global model during federated training.