Outlier
Outlier Detection and Handling is the process of identifying and managing data points that deviate significantly from the rest of the dataset — using statistical methods (Z-score, IQR), distance-based approaches (Local Outlier Factor), or isolation-based algorithms (Isolation Forest) to find anomalies that can either corrupt model training (a $10M salary when the mean is $60K) or represent the most valuable signal in the data (fraudulent transactions, equipment failures, security breaches).
What Are Outliers?
- Definition: Data points that are significantly different from the majority of observations — lying far from the center of the data distribution, potentially due to measurement errors, data entry mistakes, or genuine rare events.
- The Dual Nature: Outliers are either errors to remove or the most important data to keep. A $10M salary in an income dataset is probably a data error. A $10M transaction in a banking dataset might be fraud — the whole point of the analysis.
- Impact on Models: Linear regression is heavily influenced by outliers (a single extreme point can tilt the regression line). Tree-based models are more robust. KNN distance calculations are distorted by outliers.
Detection Methods
| Method | Approach | Assumption | Formula / Rule | ||
|---|---|---|---|---|---|
| Z-Score | Distance from mean in standard deviations | Data is roughly normal | Outlier if | z | > 3 ($z = \frac{x - \mu}{\sigma}$) |
| IQR (Interquartile Range) | Distance from median quartiles | No distribution assumption | Outlier if x < Q1 - 1.5×IQR or x > Q3 + 1.5×IQR | ||
| Isolation Forest | How easily a point can be isolated by random splits | Anomalies are rare and different | Fewer splits to isolate = more anomalous | ||
| Local Outlier Factor (LOF) | Density compared to neighbors | Outliers are in low-density regions | LOF score > 1 = lower density than neighbors | ||
| DBSCAN | Points not assigned to any cluster | Outliers are noise | Points with too few neighbors = outlier |
IQR Method Example
| Step | Calculation |
|---|---|
| Sort data | [20, 25, 28, 30, 32, 35, 38, 40, 150] |
| Q1 (25th percentile) | 26.5 |
| Q3 (75th percentile) | 39 |
| IQR = Q3 - Q1 | 12.5 |
| Lower fence = Q1 - 1.5 × IQR | 7.75 |
| Upper fence = Q3 + 1.5 × IQR | 57.75 |
| Outlier: 150 > 57.75 | ✓ Flagged |
Handling Strategies
| Strategy | Method | When to Use |
|---|---|---|
| Remove | Delete outlier rows | Measurement errors, data entry mistakes |
| Cap / Winsorize | Replace with 1st/99th percentile value | Preserve information while limiting impact |
| Transform | Log transform to reduce skew | Right-skewed distributions (income, prices) |
| Separate Model | Train different models for normal vs outlier regimes | When outliers follow different patterns |
| Keep | Leave outliers in the dataset | Fraud detection, anomaly detection (outliers ARE the target) |
| Robust Methods | Use median instead of mean, MAD instead of std | When outliers can't be removed |
Outlier Detection and Handling is the essential data quality step that protects model integrity — requiring practitioners to distinguish between errors to remove and valuable anomalies to keep, choose appropriate detection methods based on data distribution and dimensionality, and apply handling strategies that preserve the underlying signal while eliminating the noise that degrades model performance.