1. Impact Factor and Citation Metrics: What do they Really Mean? Session 1206 Room 203A January 30, 2009 Debbie Chaves, Wilfrid Laurier University Brian Cameron, Ryerson University

2. Outline Measurement of the Business of Science Background on various methods Where things can go wrong What that means for librarians

3. Why Measure? Security of knowledge Predictability Comparison analysis Subject and domain dependent Quantifiability fallacy

4. Historical Background on Impact Factor Eugene Garfield, "Citation Indexes for Science: A New Dimension in Documentation through Association of Ideas," Science 122, 3159 (July 15, 1955): 108

5. What is an Impact Factor ratio of articles published to articles cited during a rolling two-year window A/B A = citations in 2008 : articles cited in 2006-7 B = citable items published in 2006-7 year 2008 citations to 2006 + 2007 articles articles published in 2006 + 2007

6. Your (real) Impact Factor

7. Use of Impact Factors Collection development Choosing journals for publication Journal assessment/marketing by publishers Evaluation of scholarly research & individual performance, for purposes of tenure and promotion, and funding Evaluation of departments, institutions, and nations

8. Problems with IF Two-year window # of journals published in discipline ISI coverage Language Publication type Not representative Journal size

9. Other Issues Citation errors Citation clubs Citation bartering

10. Manipulation Review articles Case studies Editorial interference

11. h-index Hirsch, J. E. (2005). "An index to quantify an individual's scientific research output". PNAS 102 (46): 16569–16572.

12. h-index h-index, developed by Jorge Hirsch A scientist has index h if h of [their] N p papers have at least h citations each, and the other (N p - h) papers have at most h citations each.

14. Criticisms of the h-index Comparisons Age Insensitive Context Limitations of citation databases # of authors

15. From h to g Given a set of articles ranked in decreasing order of the number of citations that they received, the g-index is the (unique) largest number such that the top g articles received on average at least g citations.

16. What do the numbers really mean? Author inaccuracies ISI calculation inaccuracies What does an H-index of 1 really mean?

17. Citation Game-Playing – How to make it Work for YOU! Become famous so everyone gives you authorship on their paper HEP in ArXve submission timing Ride the bias – multiple authors: choose the middle Folia Phoniatrica et Logopaedica: 1 articles cited all the papers published in the last 2 years for IF=1.439 compared to 0.655 Find and ride the trend – Martin Fleischmann was once the most highly cited chemist in Britain

18. Nature 451 2008 pg 766-767

19. Search Web of Science for Wilfrid Laurier

20. Wilfrid Laurier as Author in 2007?

24. Ann Arbor as Author (MUCH worse in Google Scholar)

25. Ann Arbor as Author in 2007?

26. William H. Gates = Bill Gates

27. Researcher ID = A-1930-2009

28. Researcher ID Publication List

30. Anyone can write a letter

31. NATURE Vol 53 June 2008 pg. 718

32. Chemistry Department Scopus H-Index Created Feb 9, 2008 with the faculty at that time. Dependent on information included in SCOPUS. 09-Feb-08 Positionh-indexpatentstotal number of papers first-year publication last-year publication Publications/year president5013198420080.54 dean science15051198820082.55 assistant5014199920071.75 assistant506199820070.67 assistant15248199820075.33 assistant6013199820071.44 assistant7012199820071.33 assistant9014200020052.80 CAS304199920021.33 CAS103325197920070.89 professor8036196720070.90 associate7017199220051.31 CAS002200720082.00 CAS17024198420011.41 Average82.5019.931992.212006.141.73

33. ISI Accuracy?

35. Sciencewatch.com

36. Institutional Metrics http://webometrics.info

37. What does this mean for Librarians? Must be familiar and knowledgeable about the current calculations of Science Worthiness International students are aware Administrations and granting agencies will ask for these statistics but may not be aware of their limitations Avoid the citation culture

38. Take Home Message Avoid Disregard Syndrome Practice Citation Vigilance