Copa

COPA (Choice of Plausible Alternatives) is a commonsense reasoning benchmark — testing whether AI can identify the most plausible cause or effect given a premise, requiring understanding of everyday physical and social knowledge.

What Is COPA?

• Type: Commonsense causal reasoning benchmark.
• Task: Choose between two alternatives (cause or effect).
• Size: 1,000 questions (500 dev, 500 test).
• Focus: Everyday commonsense knowledge.
• Format: Premise + two choices, select most plausible.

Why COPA Matters

• Commonsense: Tests implicit world knowledge.
• Causal Reasoning: Requires understanding cause-effect.
• Simple Format: Clear binary choice evaluation.
• Challenging: Requires genuine understanding, not pattern matching.
• Benchmark Standard: Part of SuperGLUE evaluation suite.

Example

Premise: "The man broke his leg."
Question: What was the CAUSE?
Choice 1: "He slipped on ice." ✓
Choice 2: "He went to the hospital."

Premise: "It started raining."
Question: What was the EFFECT?
Choice 1: "People opened umbrellas." ✓
Choice 2: "The sun came out."

COPA tests commonsense causal reasoning — fundamental for human-like AI understanding.