Deformable Convolution

Deformable Convolution is a convolution with learnable spatial offsets applied to the sampling grid — allowing the kernel to sample from irregular, input-dependent positions rather than a fixed rectangular grid, adapting the receptive field to object shapes.

How Does Deformable Convolution Work?

• Standard Conv: Samples at fixed grid positions ${(-1,-1), (-1,0), ..., (1,1)}$ for a 3×3 kernel.
• Deformable: Each position gets a learned 2D offset: $p_k + \Delta p_k$ where $\Delta p_k$ is predicted by a parallel conv layer.
• Bilinear Interpolation: Since offsets are fractional, bilinear interpolation samples the feature map at non-integer positions.
• Paper: Dai et al. (2017), v2: Zhu et al. (2019).

Why It Matters

• Shape Adaptation: The receptive field adapts to object geometry — larger for large objects, deformed for non-rectangular shapes.
• Detection: Significantly improves object detection (especially for non-rigid objects) in DETR, Mask R-CNN.
• v2: Adds learnable modulation scalars to weight each sampling point's contribution.

Deformable Convolution is convolution with a flexible sampling grid — letting the network learn where to look instead of using a fixed rectangular window.