Voting

Voting Classifier

Overview
A Voting Classifier is one of the simplest ensemble learning methods. It combines the predictions of multiple distinct models to produce a final result. The core idea is that "multiple weak learners can make a strong learner" if their errors are uncorrelated.

Types of Voting

1. Hard Voting (Majority Rule)
Every model gets one vote.
- *Example*:
- Model A predicts "Spam".
- Model B predicts "Ham".
- Model C predicts "Spam".
- Result: "Spam" wins (2 vs 1).
- *Best for*: Classifiers that output discrete labels (like SVMs).

2. Soft Voting (Weighted Probabilities)
Every model outputs a probability. The final prediction is the average of these probabilities.
- *Example*:
- Model A: 0.9 Spam.
- Model B: 0.4 Spam.
- Model C: 0.8 Spam.
- Average: (0.9 + 0.4 + 0.8) / 3 = 0.7.
- Result: Spam.
- *Best for*: Well-calibrated models (Logistic Regression, Random Forest). Soft voting typically outperforms hard voting because it captures the *confidence* of the prediction.