Eca

ECA (Efficient Channel Attention) is a lightweight channel attention mechanism that captures local cross-channel interactions using a 1D convolution — avoiding the dimensionality reduction (FC bottleneck) used in SE-Net, which loses information about direct channel correspondence.

How Does ECA Work?

• Global Average Pooling: Squeeze spatial dimensions: $z \in \mathbb{R}^C$.
• 1D Convolution: Apply a 1D conv of kernel size $k$ on $z$ (captures local channel interactions).
• Adaptive $k$: $k = |\frac{\log_2 C}{\gamma} + \frac{b}{\gamma}|_{odd}$ (kernel size adapts to channel count).
• Sigmoid: Produce per-channel attention weights.
• Paper: Wang et al. (2020).

Why It Matters

• No FC Bottleneck: Avoids the information loss from SE-Net's channel reduction/expansion MLP.
• Fewer Parameters: One 1D conv layer vs. SE's two FC layers — dramatically fewer parameters.
• Same or Better Accuracy: Matches or exceeds SE-Net performance with much lower overhead.

ECA is SE-Net without the bottleneck — using a simple 1D convolution to capture channel dependencies efficiently and without information loss.