Attention Flow

Attention Flow is an interpretability technique for transformer models that computes the effective attention by propagating attention weights across layers — addressing the limitation that raw attention weights in a single layer don't capture the full information flow through a multi-layer transformer.

How Attention Flow Works

• Attention Rollout: Multiply attention matrices across layers: $A_{flow} = A_L \cdot A_{L-1} \cdots A_1$ (with residual).
• Residual Connection: Account for skip connections by adding identity matrices: $\hat{A}_l = 0.5 \cdot A_l + 0.5 \cdot I$.
• Attention Flow (Graph): Model attention as a flow network and compute max-flow from input to output tokens.
• Generic Attention: Compute the "generic" attention as the flow through the attention graph.

Why It Matters

• Multi-Layer Attribution: Raw single-layer attention can be misleading — Attention Flow captures the complete information pathway.
• Token Attribution: Shows which input tokens truly influence the output through all layers of the transformer.
• Visualization: Produces heat maps showing the effective contribution of each input token to the prediction.

Attention Flow is tracing information through the transformer — computing the effective end-to-end attention across all layers.