Graph Attention Networks

Graph Attention Networks (GAT) are neural networks that use attention mechanisms to weight neighbor importance in graphs — learning which connected nodes matter most for each node's representation, achieving state-of-the-art results on graph tasks.

What Are GATs?

• Type: Graph Neural Network with attention mechanism.
• Innovation: Learn importance weights for each neighbor.
• Contrast: GCN treats all neighbors equally, GAT weighs them.
• Output: Node embeddings incorporating weighted neighborhood.
• Paper: Veličković et al., 2018.

Why GATs Matter

• Adaptive: Learn which neighbors are important per-node.
• Interpretable: Attention weights show reasoning.
• Flexible: No fixed aggregation (unlike GCN averaging).
• State-of-the-Art: Top performance on citation, protein networks.
• Inductive: Generalizes to unseen nodes.

How GAT Works

1. Compute Attention: Score importance of each neighbor.
2. Normalize: Softmax across neighbors.
3. Aggregate: Weighted sum of neighbor features.
4. Multi-Head: Multiple attention heads, concatenate results.

Attention Mechanism

α_ij = softmax(LeakyReLU(a · [Wh_i   Wh_j]))
h'_i = σ(Σ α_ij · Wh_j)

Applications

Citation networks, protein-protein interaction, social networks, recommendation systems, molecule property prediction.

GAT brings attention to graph learning — enabling adaptive, interpretable node representations.