Tracin

TracIn (Tracing with Gradient Descent) is a data attribution method that estimates the influence of a training example on a test prediction by tracing gradient descent steps — summing the gradient alignment between training and test examples across training iterations.

How TracIn Works

• Gradient Inner Product: $TracIn(z_i, z_{test}) = \sum_t eta_t \nabla L(z_{test}, \theta_t) \cdot \nabla L(z_i, \theta_t)$.
• Checkpoints: Sum over saved training checkpoints $\theta_t$ (not every step — practical approximation).
• Learning Rate: Weight each checkpoint by the learning rate $eta_t$ at that point in training.
• Positive/Negative: Positive TracIn = training example helped the test prediction. Negative = it hurt.

Why It Matters

• Scalable: Much more practical than influence functions — no Hessian computation needed.
• Self-Influence: $TracIn(z_i, z_i)$ measures how well the model memorized training point $z_i$ — flags hard/noisy examples.
• Data Cleaning: High negative-influence training points are candidates for label errors or data quality issues.

TracIn is tracing Credit through training steps — a practical, scalable method for attributing model predictions to individual training examples.