Fouriermix

FourierMix is the spectral mixing approach that transforms features to frequency domain, applies learnable filtering, and maps back to spatial domain - by using FFT based global interactions, the model obtains full image receptive field with low computational overhead.

What Is FourierMix?

• Definition: A vision block that applies fast Fourier transform to token features, performs spectral modulation, then applies inverse FFT.
• Global Reach: Every token can influence every other token through frequency coefficients.
• Learnable Spectral Filter: Model learns which frequencies to amplify or suppress.
• Attention Alternative: Provides global mixing without explicit pairwise attention matrices.

Why FourierMix Matters

• Low Cost Global Context: FFT operations are efficient compared with quadratic attention.
• Frequency Control: Model can target low frequency semantics and high frequency detail separately.
• Noise Handling: Unwanted high frequency patterns can be attenuated in spectral space.
• Scalability: Works well for high resolution images where dense attention is expensive.
• Hybrid Flexibility: Can be combined with local convolutions or MLP channel mixers.

Spectral Block Components

FFT Transform:
- Convert spatial feature map into complex frequency coefficients.
- Preserve magnitude and phase information.

Learnable Filtering:
- Multiply coefficients by trainable weights or masks.
- Controls how each band contributes to reconstruction.

Inverse FFT:
- Return to spatial domain after spectral modulation.
- Follow with residual add and normalization.

How It Works

Step 1: Compute 2D FFT on feature map or token grid and pass frequency coefficients through learnable spectral filter layers.

Step 2: Apply inverse FFT, combine with residual path, and continue with task specific head.

Tools & Platforms

• PyTorch FFT: Native efficient fft2 and ifft2 APIs.
• CUDA kernels: Strong acceleration for batched FFT workloads.
• Hybrid backbone repos: Support plugging spectral blocks into CNN and ViT pipelines.

FourierMix is a fast global mixer that uses spectral math to connect distant regions without quadratic attention cost - it is especially useful when full context is needed at high resolution.