Context Prediction

Context Prediction is a self-supervised pretext task where the model predicts the spatial relationship between two image patches — given a center patch and a neighboring patch, the network classifies which of 8 relative positions (top-left, top, top-right, etc.) the neighbor occupies.

How Does Context Prediction Work?

• Process: Extract a center patch and one of its 8 surrounding patches. Create a gap between them to prevent trivial solutions (texture continuation).
• Classification: 8-class problem (which direction?).
• Architecture: Two-stream (each patch encoded independently) + concatenation + classifier.
• Paper: Doersch et al. (2015) — one of the earliest SSL pretext tasks.

Why It Matters

• Spatial Understanding: Learns spatial relationships and object part co-occurrence patterns.
• Pioneering: Among the first works demonstrating that self-supervised pretext tasks can learn transferable visual representations.
• Evolution: Led to jigsaw puzzles, relative patch location, and eventually modern contrastive methods.

Context Prediction is the original spatial reasoning pretext task — the granddaddy of self-supervised visual learning.