Least-to-Most Prompting
Least-to-Most Prompting is the structured prompt engineering technique that teaches language models to solve complex problems by first decomposing them into progressively simpler sub-problems, then solving from easiest to hardest — developed by Google Research as a systematic approach that significantly outperforms standard chain-of-thought prompting on tasks requiring compositional generalization, mathematical reasoning, and multi-step problem solving.
What Is Least-to-Most Prompting?
- Definition: A two-stage prompting strategy where the model first decomposes a problem into sub-problems ordered from simplest to most complex, then solves each sequentially.
- Core Innovation: Explicitly separates the decomposition step from the solving step, ensuring systematic coverage of all reasoning components.
- Key Difference from CoT: Chain-of-thought generates reasoning inline; least-to-most structures reasoning as an explicit ordered sequence of sub-problems.
- Origin: Introduced by Zhou et al. (2023) at Google Research.
Why Least-to-Most Prompting Matters
- Compositional Generalization: Enables models to solve problems more complex than any seen in few-shot examples.
- Systematic Reasoning: The ordered decomposition ensures no reasoning steps are skipped or duplicated.
- Transfer Learning: Solutions to simpler sub-problems directly inform solutions to harder ones.
- Reliability: More consistent than free-form chain-of-thought on structured problems.
- Interpretability: The explicit sub-problem chain makes reasoning fully transparent.
How It Works
Stage 1 — Decomposition:
- Present the complex problem to the model.
- Prompt the model to list sub-problems from simplest to most complex.
- Each sub-problem builds on solutions to previous simpler ones.
Stage 2 — Sequential Solving:
- Solve the simplest sub-problem first.
- Feed the solution as context for the next sub-problem.
- Continue until the most complex (original) problem is solved.
Comparison with Other Prompting Strategies
| Strategy | Decomposition | Solving Order | Context Passing |
|---|---|---|---|
| Standard Prompting | None | Direct answer | None |
| Chain-of-Thought | Implicit | Left-to-right inline | Implicit |
| Least-to-Most | Explicit, ordered | Simplest first | Explicit sub-answers |
| Tree-of-Thought | Branching | Parallel exploration | Branch-specific |
Applications & Results
- Math Word Problems: 16.2% improvement over CoT on GSM8K-style problems.
- Symbolic Reasoning: Near-perfect accuracy on last-letter concatenation tasks where CoT fails.
- Code Generation: Effective for breaking complex programming tasks into incremental steps.
- Multi-Step Planning: Natural fit for tasks requiring ordered action sequences.
Least-to-Most Prompting is a foundational advance in structured reasoning for LLMs — demonstrating that explicitly ordering sub-problems from simple to complex enables compositional generalization impossible with standard prompting approaches.