Shap

SHAP (SHapley Additive exPlanations) is the game-theoretic framework for explaining machine learning model predictions by computing each feature's fair marginal contribution to the prediction — derived from Shapley values in cooperative game theory, providing a unified, theoretically grounded explanation method applicable to any ML model.

What Is SHAP?

• Definition: A method that explains individual model predictions by assigning each input feature a Shapley value — the average marginal contribution of that feature across all possible subsets of features, measuring how much the feature shifted the prediction from the expected baseline.
• Foundation: Shapley values from cooperative game theory (Lloyd Shapley, Nobel Prize in Economics 2012) — a mathematically unique method for fairly attributing a cooperative outcome among players based on their marginal contributions.
• Analogy: Treat each feature as a "player" in a cooperative game where the "payout" is the model prediction. SHAP fairly divides credit: "Your credit score of 750 increased loan approval probability by +0.12; income of $80k added +0.08; late payment history subtracted -0.15."
• Publication: "A Unified Approach to Interpreting Model Predictions" — Lundberg & Lee, UW (2017).

Why SHAP Matters

• Theoretical Soundness: The only additive feature attribution method satisfying three mathematically proven axioms: Local Accuracy (attributions sum to prediction), Missingness (absent features get zero attribution), and Consistency (more impactful features always get higher values).
• Model-Agnostic: Works for any model — linear regression, gradient boosting, neural networks, random forests — with different computational approaches optimized for each.
• Consistent Across Methods: SHAP unifies many prior methods (LIME, DeepLIFT, LRP) — showing they are all approximations of Shapley values, providing theoretical grounding for their empirical successes.
• Global + Local Explanations: Individual Shapley values explain specific predictions; aggregating across the dataset provides global feature importance with consistent interpretability.
• Industry Standard: Deployed widely in finance (credit scoring explanation), healthcare (clinical risk model explanation), and ML platforms (Azure ML, AWS SageMaker, Google Vertex AI).

SHAP Computation Methods

KernelSHAP (Model-Agnostic, Slow):
- Approximate Shapley values by training a weighted linear model on all feature subsets.
- Theoretically exact in the limit; approximation quality depends on number of samples.
- Works for any model; slow for high-dimensional inputs (many features).

TreeSHAP (Tree Models, Fast):
- Exact Shapley values in polynomial time O(TLD²) for tree-based models (decision trees, random forests, XGBoost, LightGBM).
- Native support in XGBoost, LightGBM, CatBoost.
- Orders of magnitude faster than KernelSHAP for tree models.

DeepSHAP (Neural Networks):
- Combines DeepLIFT backpropagation with Shapley value theory.
- Approximate but fast for deep neural networks.
- Satisfies SHAP axioms approximately.

GradientSHAP:
- Combines Integrated Gradients with SHAP — samples from a distribution of baselines, averages gradients.
- Better baseline handling than single-baseline Integrated Gradients.

SHAP Visualizations

Force Plot:
- Shows how each feature's Shapley value pushes the prediction above or below the baseline.
- Red features increase prediction; blue features decrease.
- Stacked horizontally to show the complete "force" driving the output.

Summary Plot (Beeswarm):
- Each dot is one sample; x-position is Shapley value; color is feature value.
- Shows distribution of feature impacts across dataset.
- Most informative global visualization for understanding feature behavior.

Dependence Plot:
- Plot SHAP value vs. feature value for one feature.
- Reveals non-linear relationships and interaction effects.

Waterfall Plot:
- Step-by-step breakdown of a single prediction — shows exactly how each feature moved the prediction from baseline.

Shapley Value Properties

SHAP in Regulated Industries

• Credit: Explain why a loan was denied in terms of specific contributing features — complying with adverse action notice requirements (ECOA, FCRA).
• Healthcare: Show clinicians which vital signs and lab values drove a sepsis risk score — enabling clinical validation.
• Insurance: Explain premium calculations in terms of risk factors — required by insurance regulators in many jurisdictions.

SHAP Limitations

• Computational Cost: KernelSHAP requires exponentially many model evaluations; TreeSHAP is fast only for trees.
• Correlation Handling: Shapley values assume feature independence for subset sampling — correlated features can produce counter-intuitive attributions.
• Not Causal: SHAP explains model behavior, not causal relationships — high SHAP value for a feature doesn't mean changing that feature will change the outcome in the real world.

SHAP is the unified theory of feature attribution that gave machine learning explainability a mathematical foundation — by grounding explanations in 70 years of cooperative game theory, SHAP provides the principled, consistent, and auditable explanations that high-stakes AI deployment demands across every regulated industry.