1. CS626-449: NLP, Speech and Web-Topics-in-AI Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 34: Precision, Recall, F- score, Map

2. 2 Web at a glance Google indexes more than 8 billion pages Dominated by English Large part of world is deprived of this knowledge

3. 3 Search Engines Today Keyword based Irrelevant results Meaning not taken into account Language specific No search possible across language No translation possible

4. IR FRAMEWORK QUERY doc 1 doc 2.. doc n

5. SET VIEW List of Identified Entities List of Oblique Entities I O I O I ∩ O

6. MEASURES Precision, P = | I ∩ O | / | O | Recall, R = | I ∩ O | / | I | F-Score = 2 (P*R) / (P + R) F β Score = (β * P * R) / (1 + β * R)

7. OBSERVATION F-Score is a Harmonic mean of Precision and Recall HM < AM < GM Improvement in HM leads to improvement in GM and AM If no entity is left out without assigning label to it then Precision = Recall = F-Score

8. RANKED LISTS Let I and O be ranked lists. Measures:- Precision @ k = precision form list going down up to k MAP Value = (1/N) * k=1 ∑ k=N Precision @ K

9. WHAT ABOUT? R @ K = recall at K Precision at Recall R Recall at Precision P