Decision Tree Extraction

Decision Tree Extraction is a model distillation technique that trains a decision tree to approximate the predictions of a complex model — producing an interpretable tree-structured model that captures the essential decision logic of the original neural network or ensemble.

Extraction Methods

• Soft Labels: Train a decision tree using the complex model's predicted probabilities as soft targets.
• Born-Again Trees: Iteratively refine the tree using the complex model's outputs on synthetic data.
• Neural-Backed Trees: Embed neural network features into tree decision nodes for richer splits.
• Pruning: Aggressively prune to keep the tree small enough for human interpretation.

Why It Matters

• Interpretability: Decision trees are among the most interpretable model types — clear decision paths.
• Fidelity vs. Complexity: Balance between faithfully approximating the complex model and keeping the tree small.
• Regulatory: Some industries require model explanations in tree/rule form for compliance.

Decision Tree Extraction is simplifying complexity into a tree — distilling a complex model's decisions into an interpretable tree structure.