Searchqa

SearchQA is a question-answering dataset where answers must be found from multiple web search snippets, testing models' ability to aggregate evidence from noisy real-world sources.

## What Is SearchQA?

• Size: 140,000+ question-answer pairs
• Source: Jeopardy! questions with Google search snippets
• Challenge: Extract answers from 50+ noisy search results
• Context: Real web data, not curated paragraphs

## Why SearchQA Matters

Real-world QA involves searching the web, not reading a single clean document. SearchQA tests robustness to noise and evidence aggregation.

SearchQA Structure:
Question: "What is the capital of Australia?"

Search Snippets (noisy, redundant):
1. "...Sydney is the largest city in Australia..."
2. "...Canberra became the capital in 1913..."
3. "...Melbourne was briefly the capital..."
4. "...The Australian Parliament is in Canberra..."
...50+ snippets

Model must:
1. Filter irrelevant snippets
2. Aggregate evidence (Canberra appears multiple times)
3. Distinguish "largest" from "capital"
→ Answer: Canberra

SearchQA Challenges:
| Challenge | Description |
|-----------|-------------|
| Noise | Many snippets are irrelevant |
| Redundancy | Answer repeated differently |
| Distractors | Related but wrong entities |
| Length | 50+ documents to process |