1. The Journal Impact Factor Ellen Breen, Library Dublin City University

2. Overview The Thomson Reuters Journal Impact Factor Who uses it and why? Issues, influences and limitations Alternatives Conclusions

3. Journal Impact Factor (Thomson Reuters) Developed in the 60’s –Eugene Garfield and Irving Sher –To help select journals for the SCI Journal Citation Reports first produced in 1975

4. Journal Citation Reports The JCR provides quantitative tools for ranking, evaluating, categorizing, and comparing journals (Thomson Reuters) The Impact Factor is one of these tools Derived using citation data in the Science Citation Index and the Social Science Citation Index Helps determine a publication’s impact and influence in the global research community (Thomson Reuters) Widely accepted and used

5. How is the JIF calculated? 2008 JIF for Nature The number of times items published in Nature during 2006-2007 were cited in journals during 2008 divided by ____________________________________________ The number of articles published in Nature in 2006 and 2007

6. 2008 Impact Factor for Nature

7. Impact Factor Applications Used by: –Librarians –Publishers –Authors –Funding Agencies –Promotion Panels

8. How the JIF should be used Wisely! Thomson doesn’t depend on it alone to assessing the usefulness of journals so neither should anyone else It should be used with ‘informed peer review’ (lots of things influence citation rates)

9. How is the JIF used? Librarians –Collection Development and Management

10. How is the JIF used? Publishers –Marketing, market research….

11. How is the JIF used? Authors Where will I publish for maximum impact? What is the top journal in my discipline?

12. How is the JIF used? Funding Agencies Number of publications, number of citations, number of articles published in high impact journals Universities: Recruitment, Promotion, tenure

13. Impact Factor: Headlines

15. JIF: issues, influences, limitations Journal Coverage and Bias –Limited subset of journals indexed in WOS and can vary from year to year –26,000 peer-reviewed academic journals (Ulrich’s web, WOS coverage: 10,000 - 3,000 major journals account for 92% of all citations in the sciences) –Some disciplines are poorly covered –75 Nursing Journals (lobbied to increase journals covered) –Books are not included as source for citations –Bias in favour of English language titles and US publications

16. TitleImpact Factor 2008 Journal of Communication 2.266 Quarterly Journal of Economics 5.048 American Sociological Review 3.762 Communications on Pure and Applied Mathematics 3.806 Annual Review of Immunology 41.059 Nature Photonics 24.982 New England Journal of Medicine 50.017 They all have the highest impact factor in their category

17. JIF: issues, influences, limitations Discipline Differences –Different citing behaviour across disciplines e.g. Business (10-11 years before articles are cited) or few citations in some areas of Mathematics for example are not taken into account The IF cannot be used to compare journals across different subject areas “these reflect differences in disciplinary dynamics, not in quality” (Nature Editorial)

18. JIF: issues, influences, limitations 2 year IF –Favours rapidly growing fields –Short of rapid publication journals (Letters) will be cited within the 2 year window (greater immediacy) –Whereas the full paper journal tends to have a citation peak after 3 years = care when comparing journals with different mixes of article types (Amin & Mabe 2000)

19. British Journal of Radiology: Study showed that only 12% of citations to the journal in a single year were to articles published in previous 2 years 50% of cites to the BJR were to papers published in the previous 9 years. New 5 year impact factor Garfield’s own research on high- ranking journals shows the IF remains stable regardless of timeframe used

20. JIF: issues, influences, limitations 80/20 rule –Citations are not distributed equally across all articles IF alone cannot be used to assess the quality of individual articles No guarantee articles will be highly cited –Nature study found that 89% of Nature’s citations in 2004 were generated by 25% of its papers –Another study showed that a top US scientific evidence based guide cited articles from low impact journals –Articles can be cited for other reasons other than ‘a positive evaluation of its content’

21. Data: accuracy, reliability, validity Errors… Different authors with the same name….(Author ID)

22. JIF: issues, limitations Potential for manipulation –Self-citations –‘Strategic Behaviour’ to improve a journal’s IF Increase number of items that are most likely to be cited, when they appear in the journal etc Long lists of author names…. –Weakness of the IF algorithm (numerator vs denominator count)

23. JIF: issues, influences, limitations The IF Algorithm The number of times items published in Nature during 2006-2007 were cited in journals during 2008 divided by ____________________________________________ The number of ‘citable’ articles published in Nature in 2006 and 2007 (substantive items, primary research articles, reviews ) = a journal publishing lots of ‘non-citable’ items can achieve a higher IF than journals that predominantly publish ‘citable’ items = Cite your editorials!

24. JIF: issues, influences, limitations Free online availability of journal articles –Studies show that online availability can increase a journal’s IF –We can see this in the success of Open Access Journals published by PLOS and BioMed Central

25. Open Access Journals Source: 2008 JCR Science Edition

26. Open Access Journals

27. Subject Category: Mathematical and Computational Biology ‘Bioinformatics over 24 years publishing history’

28. BMC – Highly Cited Papers

29. Other methods for evaluating Journals The Eigenfactor –Applies a PageRank style algorithm like Google Weights citation from journals based on how highly cited the journal is Looks at a journal’s influence, considered influential when they are cited a lot by other influential journals –Removes self-citations (citing the same journal) –Covers 5 years of data –Available in the JCR

30. SCImago Journal Rank (SJR) Free, developed by researchers in Spain Uses Scopus citation data (more journals covered, and includes conference proceedings) Similar to Eigenfactor (PageRank algorithm) Covers 3 years of citations Excludes self-citations Includes h index of journals

31. Google Scholar Free Includes a broader range of sources (lots of non- academic sources too) Lots of errors, poor quality control Have to download and clean the data Accounting journals study found GS coverage better than Scopus and WOS

32. Article Level Metrics (former editor of the BMJ and former Chief Exec of the BMJ Publishing Group) Not just citation based: Usage is based on end-user behaviour - citations, downloads, blogs, comments, bookmarks

34. Usage Early days –Articles live in more than one place….how do you combine data? Standards –Not another number!

35. Conclusions No single ‘perfect’ JIF Objective tools have a role and can contribute to the evaluation of research quality when used appropriately – must be aware of their limitations! More data sources available to rank journals Complementary metrics (usage) should be used and studied further Evaluation: expert peer review complemented by appropriate journal ranking data Ongoing debate….

36. References Amin, M. Mabe, M. 2000. Impact factors: use and abuse. Perspectives in Publishing. 1:1-6. Corbyn, Z. A threat to scientific communication. Times Higher Education, 13 th August 2009. Dong, P. Loh, M. and Mondry, A. 2005. The “impact factor” revisited. Biomedical Digital Libraries, 2:7. Garfield, E. 2005. The agony and the ecstasy – the history and meaning of the journal impact factor. Paper presented at the International Congress on peer review and biomedical publications, Chicago, September 16 th, 2005. Ketefian, S. Freda, M.C. 2009. Impact factors and citation counts: A state of disquiet. International Journal of Nursing Studies. 46, 751-752. Reedijk, J. Moed, H.F. 2008. Is the impact of journal impact factors decreasing? Journal of Documentation. 64:2, 183-192. Rosenstreich, D. Wooliscroft. B. 2009. Measuring the impact of accounting journals using Google Scholar and the g-index. The British Accounting Review. 41:227-239. Seglen, P.O. 1997. Why the impact factor of journals should not be used for evaluating research. BMJ. 314: 498-502. Smith, R. 2006. Commentary: the power of the unrelenting impact factor – is it a force for good or harm? International Journal of Epidemiology,. 35: 1129-1130.