Condconv

CondConv (Conditionally Parameterized Convolutions) is a convolution variant where kernel weights are computed as a linear combination of expert kernels, conditioned on the input — similar to Dynamic Convolution but introduced independently by Google Brain.

How Does CondConv Work?

• Experts: $n$ convolutional kernels (experts) ${W_1, ..., W_n}$ with the same shape.
• Routing: Input-dependent routing weights $\alpha = \sigma(r(x))$ where $r$ is a routing function.
• Combined Kernel: $W = \sum_i \alpha_i W_i$.
• Apply: Standard convolution with the combined kernel.
• Paper: Yang et al. (2019).

Why It Matters

• Capacity Without Depth: Increases model capacity through kernel mixture instead of adding layers.
• Efficient Scaling: Multiple experts increase expressive power with manageable compute increase.
• EfficientNet: Used in EfficientNet-EdgeTPU architectures for mobile deployment.

CondConv is mixture-of-experts for convolution kernels — blending specialized filters based on the input for adaptive feature extraction.