Self-Explaining Neural Networks

SENN (Self-Explaining Neural Networks) are neural networks architecturally designed to produce their own explanations alongside predictions — generating interpretable concept representations and relevance scores that explain each prediction as a linear combination of meaningful concepts.

SENN Architecture

• Concept Encoder: $h(x) = [h_1(x), \ldots, h_k(x)]$ — maps input to interpretable concepts.
• Relevance Parameterizer: $\theta(x) = [\theta_1(x), \ldots, \theta_k(x)]$ — input-dependent relevance scores.
• Prediction: $f(x) = \sum_i \theta_i(x) \cdot h_i(x)$ — locally linear combination of concepts.
• Regularization: Concepts are regularized to be interpretable (sparse, coherent, diverse).

Why It Matters

• Built-In Explanation: Every prediction comes with a decomposition into concepts × relevances.
• Locally Linear: The prediction is interpretable as a locally linear model in concept space.
• No Post-Hoc: Unlike LIME/SHAP, explanations are part of the model — not approximate post-hoc attributions.

SENNs are neural networks that explain themselves — architecturally designed to decompose every prediction into interpretable components.