Spatiotemporal Detection

Spatiotemporal Detection (or Video Object Detection) is the task of tracking and classifying objects across both space and time — essentially drawing a "tube" (sequence of bounding boxes) around an object as it moves through a video.

What Is Spatiotemporal Detection?

• Goal: Detect objects in every frame and link them consistently.
• Output: A 3D volume (Tube) in the $H \times W \times T$ space.
• Challenge: Motion blur, occlusion (object disappears behind a tree), and deformation.

Why It Matters

• Autonomous Driving: Tracking pedestrians and cars is not a per-frame task; the system needs to know "Target ID 42 is moving left".
• Sports Analytics: Tracking a specific player or the ball throughout a match.
• Behavior Analysis: Understanding interactions (e.g., "Person A handed an object to Person B").

Key Datasets

• AVA (Atomic Visual Actions): Detects actions localized in space and time.
• ImageNet VID: Object detection in video.

Spatiotemporal Detection is 4D perception — understanding that objects are continuous entities that persist through time, not just flickering pixels in isolated frames.