Trajectory Convolution

Trajectory convolution is the motion-aligned convolution strategy that samples features along estimated object paths instead of fixed straight temporal tubes - this improves temporal aggregation when objects move significantly across frames.

What Is Trajectory Convolution?

• Definition: Convolution operation where temporal sampling offsets follow motion trajectories.
• Core Idea: Align receptive field with moving content to reduce motion blur in feature space.
• Difference from 3D Conv: Standard 3D kernels sample fixed positions through time.
• Input Requirement: Motion cues from optical flow or learned offset predictors.

Why Trajectory Convolution Matters

• Motion Robustness: Better feature continuity for fast-moving objects.
• Signal Quality: Reduces mixing of unrelated background pixels across frames.
• Efficiency: Focuses computation on relevant trajectories instead of dense temporal neighborhoods.
• Detection Gains: Improves recognition under camera and object motion.
• Compatibility: Can augment existing 2D or 3D convolution backbones.

Trajectory Modeling Approaches

Flow-Guided Sampling:
- Use optical flow vectors to shift sampling coordinates over time.
- Explicitly follows estimated displacement field.

Learned Deformable Offsets:
- Predict offsets end-to-end for task-specific alignment.
- Avoids explicit flow supervision.

Hybrid Schemes:
- Start with flow prior then refine with learnable offsets.
- Balances physical consistency and task optimization.

How It Works

Step 1:
- Estimate temporal motion offsets for each spatial location across adjacent frames.

Step 2:
- Apply convolution using offset sampling paths that track moving structures and aggregate aligned features.

Trajectory convolution is a motion-aware filtering method that keeps kernels locked on moving targets over time - it is especially useful when fixed temporal sampling causes heavy misalignment artifacts.