Stochastic Differential Equations

Stochastic Differential Equations (SDEs) in neural architecture are continuous-depth models that incorporate noise directly into the dynamics — $dz_t = f_\theta(z_t) dt + g_\theta(z_t) dW_t$, combining deterministic drift with stochastic diffusion for modeling uncertainty and generative processes.

SDE Neural Architecture Components

• Drift ($f_\theta$): A neural network defining the deterministic evolution direction.
• Diffusion ($g_\theta$): A neural network controlling the noise magnitude (state-dependent noise).
• Brownian Motion ($W_t$): The source of stochasticity driving the diffusion term.
• Solver: Euler-Maruyama or higher-order SDE solvers for numerical integration.

Why It Matters

• Uncertainty: Neural SDEs naturally provide uncertainty estimates through the stochastic dynamics.
• Generative Models: Score-based diffusion models and DDPM are closely related to Neural SDEs.
• Regularization: The noise acts as a continuous regularizer, improving generalization.

Neural SDEs are Neural ODEs with built-in noise — adding stochastic dynamics for uncertainty quantification and generative modeling.