Score Matching
Diffusion Model is a generative model that learns to reverse a gradual noising process — trained by predicting and removing noise step-by-step, producing state-of-the-art image, audio, and video generation.
Forward Process (Noising)
- Gradually add Gaussian noise to data over T steps (typically T=1000).
- At step T, data is pure noise: $x_T \sim N(0, I)$.
- Mathematically: $q(x_t | x_{t-1}) = N(x_t; \sqrt{1-\beta_t} x_{t-1}, \beta_t I)$
Reverse Process (Denoising)
- A neural network (usually U-Net) learns to predict the noise added at each step.
- Generation: Start from pure noise $x_T$, iteratively denoise to get $x_0$.
- The network is conditioned on timestep $t$ and optionally on a text prompt.
Key Architectures
- DDPM (Denoising Diffusion Probabilistic Models): Original formulation (Ho et al., 2020).
- DDIM: Deterministic sampling — 10-50 steps instead of 1000 (10-100x faster).
- Latent Diffusion (Stable Diffusion): Runs diffusion in compressed latent space — 8x smaller, much faster.
- Score-Based Models: Equivalent formulation using score functions $\nabla_x \log p(x)$.
Why Diffusion Models Won
- Quality: Sharper, more diverse samples than GANs.
- Stability: No adversarial training — GANs suffer from mode collapse and training instability.
- Controllability: Easy to condition on text (CLIP guidance, classifier-free guidance).
- Likelihood: Tractable likelihood computation unlike GANs.
Applications
- Image generation: DALL-E 2, Stable Diffusion, Midjourney (FLUX), Imagen.
- Video: Sora, Runway Gen-2.
- Audio: WaveGrad, DiffWave.
- Protein structure: RFDiffusion.
Diffusion models are the dominant paradigm for generative AI — they have replaced GANs across virtually every generation task and continue to advance rapidly.