Mean Average Precision (Map)

Mean Average Precision (MAP) is the average of Average Precision across multiple queries — the standard metric for evaluating search and retrieval systems across entire query sets, providing a single score for overall system performance.

What Is MAP?

• Definition: Mean of Average Precision scores across all queries.
• Formula: MAP = (Σ AP(q)) / |Q| where Q is set of queries.
• Range: 0 (worst) to 1 (perfect).

How MAP Works

1. For each query, compute Average Precision (AP).
2. Average AP scores across all queries.

Example

Query 1: AP = 0.8.
Query 2: AP = 0.6.
Query 3: AP = 0.9.
- MAP = (0.8 + 0.6 + 0.9) / 3 = 0.77.

Why MAP?

• Standard Metric: Most widely used for IR evaluation.
• Comprehensive: Evaluates entire system across all queries.
• Position-Aware: Rewards relevant results at top.
• Recall-Aware: Considers all relevant items.
• Single Score: Easy to compare systems.

MAP@K: Compute MAP considering only top-K results per query.

Advantages

• Industry Standard: Used in TREC, academic IR research.
• Comprehensive: Captures precision, recall, and position.
• Comparable: Single score for system comparison.

Limitations

• Binary Relevance: Doesn't handle graded relevance (use NDCG).
• Query Averaging: Treats all queries equally (may want weighted).
• Requires Relevance Judgments: Need labeled data for all queries.

MAP vs. Other Metrics

vs. NDCG: MAP binary relevance, NDCG graded relevance.
vs. MRR: MAP considers all relevant, MRR only first.
vs. Precision@K: MAP comprehensive, P@K single cutoff.

Applications: Search engine evaluation, information retrieval research, recommendation system evaluation, document retrieval.

Tools: trec_eval (standard IR evaluation tool), scikit-learn, IR libraries.

MAP is the gold standard for IR evaluation — by averaging precision across all relevant positions and all queries, MAP provides the most comprehensive single-number assessment of search and retrieval system quality.