Resmlp for Vision

ResMLP is the residual all-MLP architecture that simplifies Mixer style blocks with affine normalization and strong skip design for stable optimization - it aims for better data efficiency and training behavior while preserving the attention-free philosophy.

What Is ResMLP?

• Definition: An MLP based vision model that combines token interaction layers, channel MLPs, and residual blocks with lightweight normalization.
• Normalization Choice: Uses affine transforms instead of full LayerNorm in core blocks.
• Residual Emphasis: Strong identity paths keep gradients stable through deep stacks.
• Training Recipe: Heavy augmentation and regularization are important for top performance.

Why ResMLP Matters

• Optimization Stability: Residual plus affine design can converge more reliably in deep all-MLP setups.
• Data Efficiency: Often performs better than earlier Mixer variants on moderate scale datasets.
• Low Complexity: Keeps operator set small for easier deployment and profiling.
• Interpretability: Learned token-mixing weights often resemble structured spatial filters.
• Architecture Insight: Shows that normalization and residual details are as important as block type.

ResMLP Components

Token Interaction Layer:
- Mixes patch tokens with learned linear transforms.
- Works globally across the patch sequence.

Channel Feedforward Layer:
- Expands channel dimension, applies nonlinearity, then projects back.
- Supplies semantic capacity per token.

Affine Residual Wrapper:
- Applies trainable scale and shift around residual paths.
- Stabilizes updates at initialization.

How It Works

Step 1: Patchify image, project to embeddings, and run token interaction with residual addition to distribute spatial context.

Step 2: Run channel feedforward with affine scaling, repeat across stages, then pool and classify.

Tools & Platforms

• timm: Provides ResMLP variants and training scripts.
• PyTorch: Easy to customize affine and residual parameters for experiments.
• WandB: Useful for tracking sensitivity to normalization and depth.

ResMLP is a practical evolution of all-MLP vision design that trades unnecessary complexity for cleaner residual dynamics - it helps teams reach strong results with a compact and understandable architecture.