Smote
SMOTE (Synthetic Minority Over-sampling Technique) is the most widely used algorithm for handling imbalanced datasets — creating synthetic examples of the minority class by interpolating between existing minority samples rather than simply duplicating them, which expands the decision boundary and helps the model generalize to unseen minority examples instead of memorizing the few available ones, making it the standard approach for fraud detection, medical diagnosis, and any classification task where one class is dramatically underrepresented.
What Is SMOTE?
- Definition: An oversampling algorithm that creates synthetic minority class examples by selecting a minority sample, finding its K nearest minority neighbors, and generating new examples along the line segments connecting them in feature space.
- The Problem: Imbalanced datasets (99.9% legitimate transactions, 0.1% fraud) cause models to achieve high accuracy by simply predicting the majority class every time — 99.9% accuracy but 0% fraud detection.
- Why Not Just Copy?: Random oversampling (duplicating minority examples) causes overfitting — the model memorizes the exact duplicated examples. SMOTE creates new points between existing examples, expanding the minority class region.
How SMOTE Works
| Step | Process | Example |
|---|---|---|
| 1. Select minority sample A | Pick a fraud transaction | Feature vector: [amount=$500, time=2am] |
| 2. Find K nearest minority neighbors | K=5 nearest fraud points | Neighbor B: [amount=$800, time=3am] |
| 3. Pick one neighbor randomly | Choose neighbor B | |
| 4. Generate synthetic point on line A→B | Random point between A and B | New: [amount=$650, time=2:30am] |
| 5. Repeat until balanced | Continue until minority count matches majority |
Formula: $X_{new} = X_A + \lambda \times (X_B - X_A)$ where $\lambda \in [0, 1]$ is random.
SMOTE Variants
| Variant | Modification | When to Use |
|---|---|---|
| SMOTE (original) | Interpolate between any minority neighbors | General imbalance |
| Borderline-SMOTE | Only oversample minority points near the decision boundary | When boundary samples matter most |
| SMOTE-ENN | SMOTE + remove noisy samples (Edited Nearest Neighbors) | Reduce overlap after oversampling |
| SMOTE-Tomek | SMOTE + remove Tomek links (ambiguous boundary pairs) | Cleaner decision boundaries |
| ADASYN | Generate more synthetic samples for harder-to-learn minority examples | Adaptive to local difficulty |
| SMOTE-NC | Handles mixed numeric + categorical features | Datasets with categorical columns |
SMOTE vs Alternatives
| Technique | Approach | Pros | Cons |
|---|---|---|---|
| Random Oversampling | Duplicate minority examples | Simple | Overfitting on duplicates |
| SMOTE | Interpolate new minority examples | Better generalization | Can create noisy examples in overlapping regions |
| Random Undersampling | Remove majority examples | Fast, reduces data size | Loses potentially useful majority info |
| Class Weights | Increase loss penalty for minority | No data manipulation | Doesn't add new information |
| ADASYN | Adaptive SMOTE (more synthetics for harder examples) | Focuses on hard cases | More complex |
Python Implementation
from imblearn.over_sampling import SMOTE
smote = SMOTE(random_state=42, k_neighbors=5)
X_resampled, y_resampled = smote.fit_resample(X_train, y_train)
# Now minority class has same count as majorityCritical Rule: Only apply SMOTE to training data, NEVER to test/validation data. Synthetic examples in the test set would give inflated performance estimates.
SMOTE is the standard oversampling algorithm for imbalanced classification — creating synthetic minority examples through feature-space interpolation that expands the decision boundary and improves generalization, with variants like Borderline-SMOTE and SMOTE-ENN that further refine the synthetic samples for cleaner class separation.