Instancenorm

InstanceNorm (Instance Normalization) is a normalization technique that normalizes each feature map of each sample independently — computing mean and variance per channel per instance, widely used in neural style transfer and image generation.

How Does InstanceNorm Work?

• Scope: Normalize over $H \times W$ spatial dimensions for each channel of each sample independently.
• Formula: $\hat{x}_{nchw} = (x_{nchw} - \mu_{nc}) / \sqrt{\sigma_{nc}^2 + \epsilon}$
• No Batch: Statistics computed per-instance, per-channel. Completely batch-independent.
• Paper: Ulyanov et al. (2016).

Why It Matters

• Style Transfer: Removes instance-specific contrast information -> enables style transfer (AdaIN).
• Image Generation: Used in StyleGAN and other generative models for controlling per-instance statistics.
• Equivalence: InstanceNorm = GroupNorm with $G = C$ (one channel per group).

InstanceNorm is per-image, per-channel normalization — the normalization of choice for style transfer and image generation tasks.