Flownet

FlowNet is the pioneering end-to-end deep optical flow architecture that predicts dense motion directly from image pairs - it demonstrated that learned correspondence can replace many handcrafted motion-estimation stages.

What Is FlowNet?

• Definition: Convolutional encoder-decoder model for optical flow estimation from two frames.
• Original Variants: FlowNetS (stacked input) and FlowNetC (with correlation layer).
• Output: Dense per-pixel flow map at full or near-full resolution.
• Historical Impact: First major deep model family to make neural flow practical.

Why FlowNet Matters

• End-to-End Learning: Removed dependence on manual feature engineering in flow pipelines.
• Speed Gain: Enabled near real-time flow inference on GPUs at the time.
• Benchmark Shift: Sparked rapid progress in learned correspondence models.
• Architecture Foundation: Influenced PWC-Net, RAFT, and later refinement approaches.
• Practical Utility: Widely reused as initialization and baseline in motion tasks.

FlowNet Architecture Highlights

Encoder-Decoder Core:
- Downsample to capture large-context matching cues.
- Upsample with skip connections for fine motion recovery.

Correlation Module (FlowNetC):
- Explicitly compares feature patches between two frames.
- Improves matching quality for structured motion.

Multi-Scale Supervision:
- Predict flow at several scales and refine progressively.
- Stabilizes training and detail recovery.

How It Works

Step 1:
- Feed frame pair through convolutional encoder to build feature hierarchy and correspondence cues.

Step 2:
- Decode coarse-to-fine flow estimates with skip fusion to produce dense final output.

FlowNet is the foundational deep optical-flow milestone that proved neural networks can learn pixel correspondence directly from data - it remains an essential reference point in modern motion estimation history.