Shap (shapley Additive Explanations)

SHAP (SHapley Additive exPlanations) attributes prediction to input features using game-theoretic Shapley values. Core concept: From cooperative game theory - fairly distribute "payout" (prediction) among "players" (features) based on their marginal contributions. Properties: Local accuracy (sum to prediction), missingness (zero contribution for absent features), consistency (larger contribution if feature has larger effect). Computation: Exact Shapley requires 2^n feature subsets - intractable. Approximations: KernelSHAP (sampling), TreeSHAP (efficient for tree models), DeepSHAP (deep learning). For text: Each token as feature, measure contribution to prediction. Output interpretation: Positive SHAP = pushes prediction higher, negative = pushes lower. Magnitude = importance. Visualizations: Force plots, summary plots, waterfall charts. Advantages: Theoretically grounded, consistent, model-agnostic. Limitations: Expensive for text (many tokens), baseline choice matters, correlations between features complicate interpretation. Tools: shap library (Python), extensive ecosystem. Use cases: Debug models, feature importance, model comparison, compliance explanations. Industry standard for explainability.