Lookahead Decoding

Lookahead decoding is an inference acceleration technique that generates multiple tokens in parallel using speculative execution — predicting future tokens speculatively and verifying them to reduce effective latency.

What Is Lookahead Decoding?

• Definition: Generate and verify multiple tokens per forward pass.
• Method: Speculate future tokens, verify in parallel.
• Speed: 2-4× faster than standard autoregressive decoding.
• Exactness: Produces identical output to greedy decoding.
• Requirement: No additional models needed (unlike speculative decoding).

Why Lookahead Decoding Matters

• Latency: Reduces time-to-first-token and overall generation time.
• No Extra Models: Works with single model (vs speculative decoding).
• Exact: Guaranteed same output as standard decoding.
• LLM Inference: Critical for production deployments.
• Cost: More compute per step but fewer steps total.

How It Works

1. Speculate: Generate n-gram candidates for future positions.
2. Verify: Check all candidates in single forward pass.
3. Accept: Keep verified tokens, discard wrong speculations.
4. Repeat: Continue with accepted tokens.

Comparison

• Autoregressive: 1 token per forward pass.
• Speculative: Draft model + verify (needs 2 models).
• Lookahead: Self-speculate + verify (single model).

Lookahead decoding achieves faster LLM inference without auxiliary models — practical acceleration technique.