Representational Similarity Analysis

Representational similarity analysis is the method that compares geometric relationships among activations to evaluate similarity between model representations - it abstracts away from individual units to compare representational structure at scale.

What Is Representational similarity analysis?

• Definition: Builds similarity matrices over stimuli and compares matrix structure across layers or models.
• Input: Uses activation vectors from selected tokens, prompts, or tasks.
• Comparison: Similarity can be measured with correlation, cosine, or distance-based metrics.
• Output: Reveals whether two systems encode relationships among inputs in similar ways.

Why Representational similarity analysis Matters

• Cross-Model Insight: Supports architecture and checkpoint comparison without unit matching.
• Layer Mapping: Shows where representational transformations become task-aligned.
• Interpretability: Helps identify convergent or divergent encoding strategies.
• Neuroscience Link: Enables shared analysis framework across biological and artificial systems.
• Limitations: Similarity does not by itself establish causal equivalence.

How It Is Used in Practice

• Stimulus Design: Use balanced prompt sets that isolate target phenomena.
• Metric Sensitivity: Evaluate robustness across multiple similarity metrics.
• Complementary Tests: Combine RSA with intervention methods for causal interpretation.

Representational similarity analysis is a geometric framework for comparing internal representations - representational similarity analysis is most useful when geometric findings are tied to task and causal evidence.