Fedopt

FedOpt (Federated Optimization) is a framework that applies server-side adaptive optimizers (Adam, Adagrad, Yogi) to aggregate client updates — instead of simple averaging, the server uses a sophisticated optimizer to process the aggregated pseudo-gradient from client updates.

FedOpt Framework

• Client: Run local SGD as usual — send model delta $\Delta_k$ to server.
• Pseudo-Gradient: Server computes $\Delta = \sum_k p_k \Delta_k$ — the aggregated client update.
• Server Optimizer: Apply Adam/Adagrad/Yogi to this pseudo-gradient: $w_{t+1} = w_t - eta_s \cdot \text{Optimizer}(\Delta)$.
• Variants: FedAdam ($\beta_1, \beta_2$ momentum), FedAdagrad (sum of squared gradients), FedYogi (controlled adaptivity).

Why It Matters

• Better Convergence: Server-side adaptive optimization significantly improves convergence on heterogeneous data.
• Tunable: Server learning rate $eta_s$ and optimizer hyperparameters provide fine-grained control.
• State-of-Art: FedOpt variants achieve state-of-the-art federated learning performance.

FedOpt is smart server-side optimization — applying adaptive optimizers at the server to better aggregate client contributions.