Coqa

CoQA (Conversational Question Answering) is a multi-turn QA benchmark testing conversational understanding — requiring models to answer questions in context of previous dialogue turns, handling coreference, ellipsis, and follow-up questions.

What Is CoQA?

• Type: Conversational question answering benchmark.
• Task: Answer questions in multi-turn dialogue context.
• Size: 127K questions across 8K conversations.
• Domains: Children's stories, literature, Wikipedia, news, etc.
• Challenge: Handle coreference ("it", "they") and follow-ups.

Why CoQA Matters

• Conversational: Tests multi-turn dialogue understanding.
• Realistic: Mimics how humans ask follow-up questions.
• Coreference: Requires tracking entities across turns.
• Diverse: Seven different text domains.
• Standard: Used for evaluating chatbots and QA systems.

Example Dialogue

Turn 1: "Who wrote Romeo and Juliet?" → "Shakespeare"
Turn 2: "When did he write it?" → "Around 1595"
Turn 3: "What is it about?" → "Two young lovers..."

Challenges

• Resolve "he", "it", "they" to correct entities.
• Handle ellipsis ("And the next one?").
• Maintain context across many turns.

CoQA tests conversational AI capabilities — essential benchmark for dialogue systems.