Dynamic Convolution

Dynamic Convolution is a convolution where the kernel weights are dynamically generated based on the input — rather than using fixed, learned weights. The kernel adapts to each input, providing input-dependent feature extraction.

How Does Dynamic Convolution Work?

• Attention Over Kernels: Maintain $K$ fixed kernel candidates. Generate attention weights $pi_1, ..., pi_K$ from the input via squeeze-excite.
• Aggregate: $W_{dynamic} = \sum_k pi_k \cdot W_k$ (weighted sum of kernel candidates).
• Apply: Use $W_{dynamic}$ for standard convolution on the current input.
• Paper: Chen et al. (2020).

Why It Matters

• Adaptive: Different inputs get different effective kernels -> more expressive than static kernels.
• Lightweight: Only adds a small attention module to generate kernel weights.
• MobileNets: Particularly effective for lightweight models where increasing width is too expensive.

Dynamic Convolution is input-adaptive filtering — generating custom convolutional kernels on the fly for each input.