Mean Reciprocal Rank (Mrr)

Mean Reciprocal Rank (MRR) measures position of first relevant result — evaluating how quickly users find what they're looking for, with higher scores for relevant results appearing earlier in the ranked list.

What Is MRR?

• Definition: Average of reciprocal ranks of first relevant result.
• Formula: MRR = (1/|Q|) Σ (1/rank_i) where rank_i is position of first relevant result for query i.
• Range: 0 (no relevant results) to 1 (relevant result at position 1).

How MRR Works

Reciprocal Rank: 1/position of first relevant result.
- Position 1: RR = 1/1 = 1.0
- Position 2: RR = 1/2 = 0.5
- Position 3: RR = 1/3 = 0.33
- Position 10: RR = 1/10 = 0.1

MRR: Average reciprocal ranks across all queries.

Why MRR?

• User-Centric: Focuses on finding first relevant result quickly.
• Simple: Easy to understand and compute.
• Practical: Reflects real user behavior (stop at first good result).
• Question Answering: Ideal for QA where one answer suffices.

When to Use MRR

Good For: Question answering, navigational search, entity search (one correct answer).
Not Ideal For: Exploratory search, multiple relevant results, graded relevance.

MRR vs. Other Metrics

vs. NDCG: MRR only considers first relevant result, NDCG considers all.
vs. Precision@K: MRR position-aware, Precision@K counts relevant in top-K.
vs. MAP: MRR stops at first relevant, MAP considers all relevant results.

Applications: Question answering systems, entity search, navigational queries, chatbot response ranking.

Tools: Easy to implement, available in IR evaluation libraries.

MRR is perfect for single-answer scenarios — when users need one good result quickly, MRR accurately measures system effectiveness by focusing on the position of the first relevant result.