Evidence Retrieval

Evidence retrieval is the NLP task of finding documents, passages, or data that support or contradict a given claim. It is the second step in the fact-checking pipeline, connecting identified claims with the relevant information needed to verify them.

How Evidence Retrieval Works

• Query Formulation: Convert the claim into an effective search query. The claim "Global temperatures rose 1.5°C" might become a query for climate data, IPCC reports, or temperature records.
• Document Retrieval: Search large corpora (web, knowledge bases, scientific literature, fact-check archives) for relevant documents.
• Passage Extraction: Identify the specific paragraphs or sentences within retrieved documents that contain relevant evidence.
• Relevance Ranking: Rank retrieved evidence by relevance and reliability.

Retrieval Approaches

• Sparse Retrieval (BM25/TF-IDF): Traditional keyword-based search. Fast and effective for claims with distinctive terms.
• Dense Retrieval: Use neural encoders (BERT, Contriever, E5) to embed claims and documents in the same vector space, finding semantically similar evidence even without keyword overlap.
• Hybrid (Dense + Sparse): Combine keyword and semantic search using Reciprocal Rank Fusion (RRF) for better recall.
• Knowledge Graph Lookup: For claims about entities and relationships, query structured knowledge bases (Wikidata, DBpedia) directly.
• Web Search: Use search engines to find relevant web pages, especially for recent or niche claims.

Evidence Sources

• Wikipedia: Massive, structured, and frequently updated — the primary evidence source for many fact-checking systems.
• Scientific Literature: PubMed, Semantic Scholar for health and science claims.
• Government Data: Census data, economic statistics, public health records.
• Fact-Check Archives: Previously checked claims from Snopes, PolitiFact, Full Fact.
• News Archives: Verified news reports from reputable sources.

Challenges

• Source Reliability: Not all retrieved evidence is trustworthy — misinformation appears in search results too.
• Temporal Relevance: Claims about "current" statistics need up-to-date evidence, not outdated snapshots.
• Multi-Hop Reasoning: Some claims require combining evidence from multiple sources.
• Stance Detection: Determining whether retrieved evidence supports or refutes the claim adds complexity.

Evidence retrieval is the backbone of automated fact-checking — even the best verdict prediction model is useless without relevant, high-quality evidence to reason over.