Time Series Forecasting
Time Series Forecasting with Deep Learning is the application of neural network architectures to predict future values of temporal sequences — leveraging patterns in historical data including trends, seasonality, and complex nonlinear dependencies, where modern transformer and SSM-based forecasters now compete with and often surpass traditional statistical methods (ARIMA, ETS) on diverse benchmarks from energy demand to financial markets to weather prediction.
Deep Learning Architecture Timeline for Time Series
| Era | Architecture | Key Advantage |
|---|---|---|
| 2015-2017 | LSTM/GRU | Captures sequential dependencies |
| 2017-2019 | WaveNet/TCN (Temporal CNN) | Parallelizable, dilated convolutions |
| 2019-2021 | Informer/Autoformer (Transformer) | Long-range attention, multi-horizon |
| 2022+ | PatchTST, TimesNet | Channel-independent patching |
| 2023+ | TimesFM, Chronos (Foundation) | Pre-trained on many datasets |
| 2024+ | Mamba/SSM variants | Linear complexity, long sequences |
Forecasting Paradigms
| Paradigm | Method | Best For |
|---|---|---|
| Point forecast | Predict single future value at each step | Simple predictions |
| Probabilistic forecast | Predict distribution (quantiles, parameters) | Risk-aware decisions |
| Multi-horizon | Predict multiple future steps simultaneously | Planning applications |
| Multivariate | Predict multiple correlated series jointly | Interconnected systems |
PatchTST (2023)
- Key insight: Treat time series as sequence of patches (subsequences), not individual points.
- Patch size P=16: Reduces sequence length by 16x → attention cost reduced 256x!
- Channel-independent: Each variable processed independently → better scaling.
- Result: SOTA on long-term forecasting benchmarks, beating complex Transformer designs.
Foundation Models for Time Series
| Model | Developer | Approach |
|---|---|---|
| TimesFM | Pre-trained decoder-only on 100B+ timepoints | |
| Chronos | Amazon | T5-style tokenization of time series values |
| Lag-Llama | Salesforce | LLaMA-based probabilistic forecaster |
| MOIRAI | Salesforce | Universal forecaster, any-variate |
Input Representation
- Raw values: Direct numerical input → often normalized per-series.
- Patching: Group consecutive values into patches → reduce length, capture local patterns.
- Tokenization (Chronos): Bin continuous values into discrete tokens → use language model.
- Frequency features: Add day-of-week, month, hour as covariates.
- Lag features: Include values at known seasonal lags (e.g., same hour yesterday).
Evaluation Metrics
| Metric | Formula | What It Measures |
|---|---|---|
| MAE | Mean Absolute Error | Average absolute deviation |
| MSE/RMSE | (Root) Mean Squared Error | Penalizes large errors |
| MAPE | Mean Absolute Percentage Error | Scale-independent accuracy |
| CRPS | Continuous Ranked Probability Score | Probabilistic forecast quality |
| WQL | Weighted Quantile Loss | Quantile prediction accuracy |
Time series forecasting with deep learning is entering a foundation model era — pre-trained temporal models that generalize across domains are beginning to match or exceed specialized models, promising to make high-quality forecasting accessible without domain expertise, much as language models democratized NLP.