Fednova

FedNova (Federated Normalized Averaging) is a federated learning algorithm that normalizes client updates to account for different numbers of local steps — fixing the objective inconsistency in FedAvg where clients performing different amounts of local work contribute disproportionately to the global model.

How FedNova Works

• Problem: In FedAvg, a client doing 10 local steps has 10× more influence than one doing 1 step.
• Normalization: Divide each client's update by its number of local steps: $\Delta_k / \tau_k$.
• Effective Learning Rate: Normalize out the accumulated learning rate from multiple local SGD steps.
• Aggregation: Server aggregates normalized updates: $w_{t+1} = w_t - eta_g \sum_k p_k (\Delta_k / \tau_k)$.

Why It Matters

• Objective Consistency: FedNova provably converges to the correct solution, unlike FedAvg with heterogeneous local steps.
• System Heterogeneity: Clients with different compute power can run different numbers of local steps without biasing the result.
• Drop-In: Simple modification to FedAvg — just divide by local step count.

FedNova is fair averaging across unequal work — normalizing client updates to prevent faster clients from dominating the global model.