Graph Neural Network Gnn

Graph Neural Networks (GNNs) are the class of deep learning models designed to operate on graph-structured data — learning node, edge, or graph-level representations by iteratively aggregating and transforming information from neighboring nodes through message passing, enabling tasks like node classification, link prediction, and graph classification on non-Euclidean data.

Message Passing Framework:
- Neighborhood Aggregation: each node collects features from its neighbors, aggregates them, and combines with its own features — h_v^(k) = UPDATE(h_v^(k-1), AGGREGATE({h_u^(k-1) : u ∈ N(v)})); k layers enable each node to incorporate information from k-hop neighbors
- Aggregation Functions: sum, mean, max, or learnable attention-weighted aggregation — choice affects model's ability to distinguish graph structures; sum aggregation is maximally expressive (can count neighbor features)
- Update Functions: linear transformation followed by non-linearity — W^(k) × CONCAT(h_v^(k-1), agg_v) + b^(k) with ReLU/GELU activation; residual connections added for deeper networks
- Readout (Graph-Level): aggregate all node representations for graph-level prediction — sum, mean, or hierarchical pooling across all nodes; attention-based readout learns which nodes are most important for the graph-level task

Key GNN Architectures:
- GCN (Graph Convolutional Network): spectral-inspired convolutional operation — h_v^(k) = σ(Σ_{u∈N(v)∪{v}} (1/√(d_u × d_v)) × W^(k) × h_u^(k-1)); symmetric normalization by degree prevents high-degree nodes from dominating
- GAT (Graph Attention Network): attention-weighted neighbor aggregation — attention coefficients α_vu = softmax(LeakyReLU(a^T[Wh_v Wh_u])) learned per edge; multi-head attention analogous to Transformer attention; dynamically weights neighbors by importance
- GraphSAGE: samples fixed number of neighbors and aggregates using learned function — enables inductive learning (generalizing to unseen nodes/graphs at inference); mean, LSTM, or pooling aggregators
- GIN (Graph Isomorphism Network): provably maximally expressive under the Weisfeiler-Leman framework — uses sum aggregation with MLP update: h_v^(k) = MLP((1+ε) × h_v^(k-1) + Σ h_u^(k-1)); distinguishes more graph structures than GCN/GraphSAGE

Applications and Challenges:
- Molecular Property Prediction: atoms as nodes, bonds as edges — GNNs predict molecular properties (toxicity, binding affinity, solubility) directly from molecular graphs; SchNet and DimeNet incorporate 3D geometry
- Recommendation Systems: users and items as nodes, interactions as edges — GNN-based collaborative filtering (PinSage, LightGCN) captures multi-hop user-item relationships for better recommendations
- Over-Smoothing: deep GNNs (>5 layers) produce nearly identical node representations — all nodes converge to the same embedding as neighborhood expands to cover entire graph; solutions: residual connections, jumping knowledge, DropEdge regularization
- Scalability: full-batch GNN training on large graphs requires O(N²) memory — mini-batch training (GraphSAINT, Cluster-GCN) samples subgraphs; neighborhood sampling (GraphSAGE) limits per-node computation

Graph neural networks extend deep learning beyond grid-structured data to the rich world of relational and structural information — enabling AI systems to reason about molecules, social networks, knowledge graphs, and any domain where entities and their relationships form the natural data representation.