1 Relevance Ranking in the Scholarly Domain Dr. Tamar Sadeh LIBER Conference Tartu, Estonia, June 2012 Dr. Tamar Sadeh LIBER Conference Tartu, Estonia, June 2012

2

The top three keys for success

1 1 Content

2 2 Speed

3 3 Relevance Ranking

8 Relevance is the measure of correspondence between a document and a query as determined by a user Based on Saracevic, 1975

9 ? ?

10 System or algorithmic relevance Topical or subject relevance Cognitive relevance or pertinence Situational relevance or utility Affective relevance System or algorithmic relevance Topical or subject relevance Cognitive relevance or pertinence Situational relevance or utility Effective relevance

11 There is no absolute relevance

12

13

14

15

16

17

18 The ScholarRank Project

19 The Goal Enhance the Primo relevance ranking algorithm The Goal Enhance the Primo relevance ranking algorithm

20 Relevance ranking was not new to us.

28 Methodology

29 Setting up a team Building test environment, tools, and procedures Defining metrics to evaluate our current success and the improvements we make Defining measurements to assess the success of the changes, once implemented

30 Evaluation

31 Working with researchers Researchers’ evaluation quantified Enhancements introduced and checked in the lab, using defined metrics Enhancements launched and usage patterns monitored Improvements are introduced on an ongoing basis

32 How is relevance ranking calculated?

33 ? ?

34 abstract author date full text journal language type publisher subject title citations downloads journal impact factor eigenfactor pagerank

35 academic degree discipline(s) language location previous selections search history

36 broad-topic search currency exact-item search material type narrow-topic search ? ?

37 Broad-topic query Narrow-topic query Author-related query Known-item query

? ?

39 ? ?

40 The match: traditional information retrieval methods, adapted to the scholarly environment

43 ? ? no. of citations; no. of selections; recency; type; peer review

46 ? ? Academic degree, discipline

47

48 Computer Science

49 Psychology

50 Mathematics

51 Physics

52 ? ? Author-related query, known-item query, broad-topic query…

53 Before

54 After Tachycardia – Wikipedia the free encyclopedia Tachycardia comes from the Greek words tachys (rapid or accelerated) and kardia (of the heart). Tachycardia typically refers to a heart rate that exceeds the normal range for a resting...

55

56 ? ? ScholarRank™

57

58 Thank You! Tamar Sadeh, PhD Tamar Sadeh, PhD