Scratchpad

Scratchpad prompting is the technique of providing the language model with a designated workspace area where it can show intermediate calculations, working notes, and step-by-step reasoning before producing a final answer — mimicking how humans use scratch paper to work through complex problems.

Why Scratchpads Help

• Without a scratchpad, the model must compute everything "in its head" — maintaining intermediate results in its hidden state, which is prone to errors for multi-step problems.
• A scratchpad externalizes working memory — the model writes down intermediate results as text tokens, which then become part of the visible context for subsequent reasoning.
• This is especially important for arithmetic, symbolic manipulation, and multi-step logic where tracking intermediate values is critical.

Scratchpad Format

Question: What is 47 × 83?

Scratchpad:
47 × 83
= 47 × 80 + 47 × 3
= 3760 + 141
= 3901

Answer: 3901

Scratchpad vs. Chain-of-Thought

• Chain-of-Thought: Natural language reasoning narrative — "First, I note that... then I consider... therefore..."
• Scratchpad: More structured, often using notation, symbols, and compact working — closer to how you'd write on actual scratch paper.
• Overlap: Both externalize reasoning. Scratchpad tends to be more compact and calculation-focused. CoT tends to be more narrative and explanation-focused.
• In practice, they're often combined — natural language reasoning with interspersed calculations.

When Scratchpads Are Most Effective

• Arithmetic: Multi-digit multiplication, division, compound calculations — the model writes out partial products and carries.
• Symbolic Manipulation: Algebra, equation solving, simplification — each transformation step written explicitly.
• Code Tracing: Stepping through code execution — tracking variable values at each line.
• Logic Problems: Truth tables, constraint tracking, elimination — writing out what's known and what's ruled out.
• State Tracking: Problems involving changing state (puzzles, simulations) — recording state after each action.

Scratchpad Training

• Few-Shot: Include scratchpad demonstrations in the prompt — the model learns to use the scratchpad format from examples.
• Fine-Tuning: Models fine-tuned on data with scratchpad traces learn to produce scratchpads without explicit prompting.
• Verifier Training: A separate model can be trained to check the scratchpad work — identifying errors in intermediate steps.

Benefits

• Accuracy: Scratchpads can improve math accuracy by 20–50% on complex calculations compared to direct answering.
• Debuggability: When the answer is wrong, you can inspect the scratchpad to find exactly where the error occurred.
• Reproducibility: The explicit working makes the reasoning transparent and reproducible.

Practical Tips

• Explicitly instruct: "Use a scratchpad to show your work before giving the final answer."
• For few-shot prompting, include examples with scratchpad work shown.
• Keep the scratchpad focused — too much extraneous work can distract the model from the core calculation.

Scratchpad prompting is a simple but powerful technique — by giving the model space to show its work, it transforms error-prone mental computation into reliable, step-by-step written reasoning.