Representer Point Selection

Representer Point Selection is a data attribution technique that decomposes a model's prediction into a linear combination of training example contributions — expressing the pre-activation output as $\sum_i \alpha_i k(x_i, x_{test})$ where $\alpha_i$ quantifies training point $i$'s contribution.

How Representer Points Work

• Representer Theorem: For L2-regularized models, the pre-activation prediction decomposes into training point contributions.
• Weight $\alpha_i$: $\alpha_i = -\frac{1}{2\lambda n} \frac{\partial L}{\partial f(x_i)}$ — proportional to the gradient of the loss at that training point.
• Kernel: $k(x_i, x_{test}) = \phi(x_i)^T \phi(x_{test})$ in the feature space of the penultimate layer.
• Ranking: Sort training points by $\alpha_i \cdot k(x_i, x_{test})$ to find the most influential examples.

Why It Matters

• Decomposition: Every prediction is explicitly decomposed into training example contributions.
• Proponents/Opponents: Positive contributions are proponents (support the prediction); negative are opponents.
• Interpretable: Shows which training examples the model "relies on" for each prediction.

Representer Points are predictions explained by training examples — decomposing every output into specific contributions from individual training data.