Neural Controlled Differential Equations

Neural CDEs are a neural network architecture that parameterizes the response function of a controlled differential equation with a neural network — $dz_t = f_\theta(z_t) , dX_t$, providing a continuous-time, theoretically grounded model for irregular time series classification and regression.

How Neural CDEs Work

• Input Processing: Interpolate the irregular time series ${(t_i, x_i)}$ into a continuous path $X_t$.
• Neural Response: $f_\theta$ is a neural network mapping the hidden state to a matrix that interacts with $dX_t$.
• ODE Solver: Solve the CDE using standard adaptive ODE solvers (Dormand-Prince, etc.).
• Output: Read out the prediction from the terminal hidden state $z_T$.

Why It Matters

• Irregular Time Series: Purpose-built for irregularly sampled data — outperforms RNNs, LSTMs, and Transformers on irregular benchmarks.
• Missing Data: Naturally handles missing channels and variable-length sequences.
• Memory Efficient: Adjoint method enables constant-memory training regardless of sequence length.

Neural CDEs are continuous RNNs for irregular data — using controlled differential equations to process time series with arbitrary sampling patterns.