Cost-Sensitive Learning

Cost-Sensitive Learning is a machine learning framework that incorporates different misclassification costs for different classes or types of errors — using a cost matrix to penalize certain errors more heavily, reflecting the real-world consequences of different types of misclassifications.

Cost-Sensitive Methods

• Cost Matrix: Define costs for each (true class, predicted class) pair — not all mistakes are equal.
• Weighted Loss: Weight the loss function by class-specific costs: $L = \sum_i c(y_i, \hat{y}_i) \cdot ell(y_i, \hat{y}_i)$.
• Threshold Adjustment: Modify the decision threshold based on the cost ratio.
• Meta-Learning: Learn the cost weights from validation performance.

Why It Matters

• Asymmetric Costs: Missing a killer defect (false negative) is far more costly than a false alarm (false positive).
• Business Alignment: Costs can reflect actual financial impact of each error type.
• Flexible: Cost-sensitive learning is model-agnostic — applies to any classifier.

Cost-Sensitive Learning is pricing each mistake — incorporating the real-world cost of different errors into the model's training objective.