Fill in Middle
Fill-in-the-Middle (FIM) is a training objective and inference capability that enables code models to complete missing code given both the text before and after the cursor position — solving the fundamental limitation of standard left-to-right language models that can only see preceding context, by training models to predict masked spans using both prefix (code above cursor) and suffix (code below cursor), making it the essential technique powering production IDE features like Copilot's inline completions where the model must generate code that fits coherently between existing code blocks.
What Is FIM?
- Definition: A training technique where code spans are randomly extracted from their original position and relocated to the end of the training sequence — the model learns to reconstruct the missing middle given both the prefix (text before the gap) and suffix (text after the gap).
- The Problem with Standard LLMs: Causal (left-to-right) language models only see text before the cursor. When a developer's cursor is inside a function with code both above and below, a standard model cannot use the code below to inform its suggestion — leading to completions that conflict with what comes next.
- FIM Training Format: During training, a random span is extracted and the sequence is rearranged:
Original: def add(a, b): result = a + b; return result
FIM Format: <PRE> def add(a, b): <SUF> return result <MID> result = a + b;The model learns that `<MID>` is the content that goes between `<PRE>` and `<SUF>`.
Why FIM Matters for IDE Completions
| Scenario | Without FIM | With FIM |
|---|---|---|
| Cursor inside function body | Only sees code above — may conflict with return statement below | Sees code above AND below — generates consistent completion |
| Editing between existing lines | Unaware of following code | Adapts to match surrounding context |
| Adding to middle of class | Ignores existing methods below | Generates code consistent with class structure |
| Infilling function arguments | Only sees function name | Sees both function signature and body |
Models Trained with FIM
| Model | FIM Support | FIM Training | Performance Impact |
|---|---|---|---|
| Code Llama | Yes (all variants) | 7% of training data | +15% on infilling benchmarks |
| StarCoder | Yes | 50/50 prefix-suffix-middle | Significant improvement |
| DeepSeek Coder | Yes | Repository-level FIM | State-of-the-art infilling |
| InCoder (pioneer) | Yes (pioneered FIM) | Full FIM training | First model to demonstrate FIM |
| Codestral | Yes | Optimized FIM | Production IDE focus |
FIM is the indispensable training technique that enables practical IDE code completion — by teaching models to predict missing code given both surrounding context, FIM transforms language models from left-to-right text generators into bidirectionally-aware coding assistants that generate completions fitting seamlessly into existing codebases.