Label Encoding
Label Encoding is a simple categorical encoding technique that assigns a unique integer to each category — mapping "Red" → 0, "Green" → 1, "Blue" → 2 — providing a compact representation that is appropriate for ordinal data (Low < Medium < High) and tree-based models (which split on thresholds regardless of ordinal meaning), but problematic for linear models and distance-based algorithms that interpret the integers as having mathematical relationships (2 > 1 > 0 implies an ordering that may not exist).
What Is Label Encoding?
- Definition: A mapping from categorical string values to integer values — each unique category receives a unique integer, typically assigned alphabetically or in order of appearance.
- When It's Correct: For ordinal variables where the order matters (education: High School=0, Bachelor's=1, Master's=2, PhD=3) or satisfaction ratings (Low=0, Medium=1, High=2).
- When It's Dangerous: For nominal (unordered) variables — encoding City as New York=0, London=1, Tokyo=2 implies London is "between" New York and Tokyo numerically, which is meaningless.
Label Encoding Example
| Original | Encoded |
|---|---|
| "Cat" | 0 |
| "Dog" | 1 |
| "Fish" | 2 |
| "Cat" | 0 |
| "Fish" | 2 |
When to Use Label Encoding
| Scenario | Safe? | Reason |
|---|---|---|
| Ordinal features (Low/Medium/High) | Yes ✓ | Order is meaningful |
| Tree-based models (Random Forest, XGBoost) | Yes ✓ | Trees don't assume ordinal meaning, they just find optimal split thresholds |
| Linear Regression with nominal features | No ✗ | Model learns weight × integer, implying order |
| KNN / SVM with nominal features | No ✗ | Distance calculations treat integers as ordered |
| Neural Networks with nominal features | No ✗ | Embedding layers or one-hot are preferred |
| Target variable encoding | Yes ✓ | sklearn requires numeric targets for classification |
Label Encoding vs Alternatives
| Encoding | # Columns | Ordinal Assumption | High Cardinality | Best For |
|---|---|---|---|---|
| Label Encoding | 1 (same column) | Yes (implied) | Handles well | Ordinal features, tree models, target labels |
| One-Hot Encoding | K (one per category) | No | Explodes dimensionality | Linear models, neural networks |
| Target Encoding | 1 (continuous) | No | Handles well | High-cardinality + supervised learning |
| Ordinal Encoding | 1 (explicit order mapping) | Yes (explicit) | Handles well | When you define the order |
Python Implementation
from sklearn.preprocessing import LabelEncoder, OrdinalEncoder
# LabelEncoder (for target variable / single column)
le = LabelEncoder()
y_encoded = le.fit_transform(["cat", "dog", "fish", "cat"])
# [0, 1, 2, 0]
# OrdinalEncoder (for features with custom order)
oe = OrdinalEncoder(categories=[["low", "medium", "high"]])
X_encoded = oe.fit_transform(df[["satisfaction"]])Label Encoding is the compact, memory-efficient encoding for ordinal features and tree-based models — providing a single-column integer representation that preserves ordering information, with the critical caveat that it should never be used for nominal (unordered) categories in linear or distance-based models where the implied ordinal relationship corrupts the model's learning.