Iterative Refinement

Iterative Refinement in text generation is a strategy where the model generates an initial output and then repeatedly refines it through multiple passes — each iteration improves upon the previous output by correcting errors, filling in masked positions, or adjusting token choices, converging toward a high-quality final result.

Iterative Refinement Methods

• Mask-Predict: Mask the least confident tokens from the previous iteration — re-predict them conditioned on the rest.
• CMLM (Conditional Masked Language Model): Ghazvininejad et al. — iteratively unmask tokens from a fully masked initial sequence.
• Edit-Based: Identify and modify specific positions — insertions, deletions, and replacements.
• Denoising: Add noise to the previous output and denoise — each iteration removes more noise.

Why It Matters

• Quality Recovery: Recovers much of the quality gap between non-autoregressive and autoregressive models.
• Adaptive Compute: More iterations = better quality — can stop early for speed or continue for quality.
• Flexible: Works with various base architectures — Transformer, diffusion models, or edit-based models.

Iterative Refinement is draft and polish — generating an initial output and progressively improving it through multiple correction passes.