Test Time Compute
Test-Time Compute Scaling is the paradigm of allocating more computational resources at inference time to improve output quality — contrasting with training-time scaling (more data/parameters) by spending more FLOPS per query to achieve better answers.
The Core Insight
- Training scaling: 10x more compute → 10x better model (Chinchilla law).
- Inference scaling: Generate N answers → select best → improves accuracy without retraining.
- Key finding (Snell et al., 2024): "Beyond the chinchilla optimum, test-time compute is more efficient than training compute for difficult tasks."
Test-Time Compute Methods
Best-of-N Sampling:
- Generate N independent responses → select best by reward model score.
- Simple but effective. O(N) compute. Linear in N, but diminishing returns.
Sequential Refinement:
- Generate → self-critique → revise → repeat K times.
- Each iteration improves quality, especially for complex tasks.
Monte Carlo Tree Search (MCTS):
- Expand reasoning tree, evaluate leaf nodes with process reward model.
- Backpropagate scores → select best reasoning path.
- AlphaGo approach applied to language reasoning.
OpenAI o1 and "Chain of Thought":
- o1 generates an internal "thinking chain" before answering — extended CoT.
- More thinking tokens → better accuracy (log-linear relationship).
- o1: 83.3% on AIME 2024 (vs. GPT-4o: 9.3%).
- o3: >90% on ARC-AGI challenge with heavy test-time compute.
Scaling Laws for Inference
- Accuracy vs. compute: ~log-linear on difficult reasoning benchmarks.
- Crossover point: For hard tasks, spending 10x inference compute beats training a 10x larger model.
- Cost implication: Test-time compute shifts cost from upfront (training) to per-query.
Efficient Test-Time Compute
- Adaptive compute: Allocate more compute for harder questions, less for easy.
- Speculative thinking: Draft short CoT; extend only if initial answer uncertain.
Test-time compute scaling is the new frontier of AI capability improvement — the o1/o3 results show that reasoning quality can be traded against compute budget, opening a new axis of scaling beyond model size and training data.