Snail

SNAIL (Simple Neural Attentive Learner) is a meta-learning architecture that uses temporal convolutions and attention to aggregate experience — processing a sequence of observations and labels (or states and rewards) to make predictions for new inputs, combining the local focus of convolutions with the global access of attention.

SNAIL Architecture

• Temporal Convolutions: Causal dilated convolutions capture local temporal patterns in the experience sequence.
• Attention: Soft attention over all previous experiences — enables global access to any past observation.
• Interleaved: Alternate convolution and attention blocks — convolutions provide features, attention retrieves relevant memories.
• Sequence: The entire support set is processed as a sequence — each new query can attend to all past examples.

Why It Matters

• General: Works for both supervised few-shot learning and meta-RL — a unified architecture.
• Scalable: Attention handles variable-length experience — no fixed context window.
• Structure: Temporal convolutions capture local structure that pure attention might miss.

SNAIL is the attention-based meta-learner — combining temporal convolutions and attention to learn from sequential experience for fast adaptation.