Video Question Answering

Video Question Answering (VideoQA) is a multimodal task where a model answers natural language questions about a video clip — requiring the integration of visual features, temporal dynamics, and linguistic semantics to understand events that unfold over time.

What Is VideoQA?

• Definition: Given Video $V$ and Question $Q$, predict Answer $A$.
• Complexity: Unlike ImageQA, the answer often depends on *when* something happens or the *sequence* of actions.
• Types:
• Descriptive: "What is the man doing?"
• Temporal: "What did he do after opening the door?"
• Causal: "Why did the car stop?"

Why It Matters

• Search: "Find the part of the meeting where we discussed the budget."
• Surveillance: "Did anyone enter the room between 2 PM and 3 PM?"
• Accessibility: Helping visually impaired users understand dynamic content.

Key Challenges

• Long-Term Dependency: Remembering details from the start of a long video to answer a question at the end.
• Multimodal Fusion: aligning audio (speech), visual frames, and motion information.

Video Question Answering is comprehension for dynamic scenes — testing an AI's ability to maintain a coherent mental model of a changing world.