Variance-Exploding Diffusion
Variance-exploding diffusion is the score-based diffusion process where noise variance expands strongly over time while clean signal scaling is handled differently - it is common in continuous-time score modeling and sigma-parameterized formulations.
What Is Variance-exploding diffusion?
- Definition: State variance increases from low sigma to high sigma across diffusion time.
- Modeling Style: Networks often predict score or denoising direction conditioned on sigma levels.
- Continuous Form: Frequently expressed as a VE SDE rather than a discrete DDPM chain.
- Sampling: Requires integrators aware of sigma-space dynamics and noise scaling.
Why Variance-exploding diffusion Matters
- Coverage: Strong high-noise regime can improve robustness of score estimation.
- Flexibility: Useful alternative when VP assumptions are not ideal for the data domain.
- Theoretical Link: Connects naturally to score-matching views of generative modeling.
- Design Diversity: Expands sampler and architecture options beyond VP-only pipelines.
- Tradeoff Awareness: Can demand careful preconditioning to maintain stable optimization.
How It Is Used in Practice
- Sigma Grid: Choose sigma_min and sigma_max ranges that match dataset dynamic range.
- Preconditioning: Use input-output scaling schemes tailored for wide sigma intervals.
- Solver Choice: Select samplers validated on VE SDEs instead of reusing VP defaults blindly.
Variance-exploding diffusion is an important continuous-time alternative to VP diffusion parameterization - variance-exploding diffusion performs best with sigma-aware training and sampler design.