Zero-Shot Chain-of-Thought

Zero-shot chain-of-thought (Zero-shot CoT) is the remarkably simple technique of appending the phrase "Let's think step by step" (or a similar instruction) to a prompt — without providing any reasoning examples — to trigger the language model to generate its own step-by-step reasoning before producing a final answer.

The Discovery

• Standard few-shot CoT requires carefully crafted reasoning examples in the prompt — effective but labor-intensive to create for each task.
• Researchers discovered that simply adding "Let's think step by step" to the end of a zero-shot prompt (no examples at all) dramatically improves reasoning performance.
• This single phrase can improve accuracy on math and logic tasks by 40–70% compared to standard zero-shot prompting.

How Zero-Shot CoT Works

• Without CoT: "What is 23 + 47 × 2?" → Model often gives wrong answer by misapplying order of operations.
• With Zero-Shot CoT: "What is 23 + 47 × 2? Let's think step by step." → Model responds:

Step 1: First, compute 47 × 2 = 94
  Step 2: Then, add 23 + 94 = 117
  Answer: 117

Two-Stage Process

1. Reasoning Extraction: Append "Let's think step by step" → model generates a reasoning chain.
2. Answer Extraction: After the reasoning, prompt "Therefore, the answer is" → model produces the final answer.
- Some implementations use both stages explicitly; others let the model naturally conclude with an answer.

Why It Works

• The phrase activates reasoning patterns learned during pretraining — the model has seen many examples of step-by-step reasoning in its training data.
• Without the prompt, the model defaults to pattern matching or direct recall — which often fails for problems requiring multi-step logic.
• The instruction makes the model allocate more computation (more tokens) to the problem before committing to an answer.

Effective Trigger Phrases

• "Let's think step by step" — the original and most studied.
• "Let's work this out step by step to be sure we have the right answer."
• "Let's solve this carefully."
• "Think about this step by step before answering."
• Research shows the exact phrasing matters — some variations work better than others for specific models.

Limitations

• Less Effective Than Few-Shot CoT: On many benchmarks, few-shot CoT with well-crafted examples still outperforms zero-shot CoT.
• Model Size Dependent: Zero-shot CoT primarily works with large models (>100B parameters). Smaller models may produce incoherent reasoning.
• Task Dependent: Works well for math, logic, and commonsense reasoning. Less effective for creative tasks or tasks requiring domain-specific procedures.
• Unfaithful Reasoning: The model may generate plausible-looking but logically flawed reasoning — the presence of steps doesn't guarantee correctness.

Practical Impact

• Zero-shot CoT is the most cost-effective reasoning improvement available — it requires no example crafting, no fine-tuning, and works across many tasks.
• It's become a standard baseline in prompt engineering — virtually every complex prompt now includes some form of "think step by step" instruction.

Zero-shot chain-of-thought is one of the most influential discoveries in prompt engineering — a single phrase that unlocks latent reasoning capabilities, demonstrating that how you ask is as important as what you ask.