Triviaqa

TriviaQA is a large-scale reading comprehension dataset containing over 650k question-answer-evidence triples — derived from trivia enthusiasts' websites, it features complex, compositional questions that often require reasoning across multiple sentences or documents.

Characteristics

• Distant Supervision: Evidence documents are gathered automatically from Bing search results, not manually paired.
• Complexity: Questions are authored by trivia buffs, so they are harder, punnier, and more nuanced than SQuAD.
• Length: Context documents are full web/wiki pages, much longer than SQuAD paragraphs.

Why It Matters

• Long Context: Tests the model's ability to filter relevant info from large amounts of noise.
• World Knowledge: High performance correlates with the model's internal knowledge base (common in LLMs).
• Open Domain: Often used in the "Closed Book" setting (answer without seeing the document) to test model memory.

TriviaQA is pub quiz for AI — complex, nuanced questions requiring broad world knowledge and deep reading comprehension.