Deconvolution Networks

Deconvolution Networks (DeconvNets) are a visualization technique that projects feature activations back to the input pixel space — using an approximate inverse of the convolutional network to reconstruct what input pattern caused a particular neuron or feature map activation.

How DeconvNets Work

• Forward Pass: Run the input through the CNN, record activations at the layer of interest.
• Set Target: Zero out all activations except the neuron(s) to visualize.
• Backward Projection: Pass through "deconvolution" layers — transpose conv, unpooling (using switch positions), ReLU.
• ReLU Handling: Apply ReLU in the backward pass based on the sign of the backward signal (not the forward activation).

Why It Matters

• Feature Understanding: Visualize what each neuron in the CNN has learned to detect.
• Debugging: Identify neurons that detect artifacts, noise, or irrelevant features.
• Historical: Zeiler & Fergus (2014) — one of the first systematic approaches to understanding CNN features.

DeconvNets are the CNN's projector — projecting internal feature activations back to pixel space to reveal what patterns each neuron detects.