Raft

RAFT is the high-accuracy optical flow architecture that uses all-pairs correlation and recurrent iterative updates to refine motion estimates - instead of coarse-to-fine shrinking of search space, it repeatedly optimizes flow on a fixed high-resolution field.

What Is RAFT?

• Definition: Recurrent All-Pairs Field Transforms model for dense optical flow.
• Core Feature: 4D correlation volume containing pairwise matching scores for all pixel locations.
• Update Mechanism: GRU-like recurrent unit performs many refinement iterations.
• Resolution Strategy: Maintains a single flow field scale and improves it step by step.

Why RAFT Matters

• State-of-the-Art Accuracy: Strong benchmark performance on challenging flow datasets.
• Refinement Stability: Iterative updates correct errors gradually and robustly.
• Fine Detail Recovery: Handles small structures and thin boundaries better than many older methods.
• Generalization Strength: Performs well across synthetic and real-world motion domains.
• System Impact: Became a dominant flow backbone for many downstream tasks.

RAFT Architecture

All-Pairs Correlation:
- Precompute dense correspondence tensor between frame feature maps.
- Provide rich search space for iterative updates.

Recurrent Update Block:
- Query correlation pyramid and current flow to predict residual update.
- Repeat for fixed number of iterations.

Context Encoder:
- Extract static scene context to guide recurrent optimization.
- Improves convergence and boundary precision.

How It Works

Step 1:
- Encode both frames, build all-pairs correlation volume, initialize flow field.

Step 2:
- Run recurrent update loop to refine flow iteratively until convergence.

RAFT is a refinement-centric optical flow model that turns dense correspondence into precise motion estimates through iterative optimization - it set a new standard for high-quality flow prediction in modern video pipelines.