Iterated Amplification

Iterated Amplification is an AI alignment technique that bootstraps human oversight by iteratively using AI assistance to solve increasingly complex evaluation tasks — starting with problems humans can evaluate directly, then using AI-assisted humans to evaluate slightly harder problems, and continuing to expand the frontier of evaluable tasks.

Amplification Process

• Base Case: Human evaluates simple AI outputs directly — standard RLHF.
• Amplification Step: For harder tasks, decompose into sub-problems that a human-with-AI-assistant can evaluate.
• Iteration: The AI assistant itself was trained using the previous round's amplified evaluator.
• Distillation: Train a new model to mimic the amplified evaluator — producing a standalone, efficient model.

Why It Matters

• Scalable Oversight: Enables evaluation of AI outputs that are too complex for unaided human judgment.
• Alignment Path: Provides a concrete path to aligning superhuman AI — evaluation capability grows with AI capability.
• Decomposition: Complex tasks are decomposed into human-manageable sub-problems — divide and conquer for alignment.

Iterated Amplification is growing the evaluator alongside the AI — bootstrapping human oversight to keep pace with increasingly capable AI systems.