Background Briefings for Librarians #1: Impact Factor Paola Gargiluo speaking with Dr. David F. Kohl CASPUR Podcast Series May 28, 2008

Dr. David F. Kohl Began professional library work in 1972 Began professional library work in 1972 –Dean Emeritus, University of Cincinnati –Past President, Reference and Adult Users Association of the American Library Association –Senior Fulbright Scholar  Turkey…Taiwan…Greece…India… –Founder, U. of Cincinnati Digital Press Retired, but still active as: Retired, but still active as: –International Library Consultant –Editor in Chief, Journal of Academic Librarianship

What is Journal Impact Factor (JIF) and why has it become important? There is a new sense of accountability in Universities and JIF provides an important measure of the quality of faculty productivity There is a new sense of accountability in Universities and JIF provides an important measure of the quality of faculty productivity –External –Objective JIF (a single number) is quick, easy, and convenient to use JIF (a single number) is quick, easy, and convenient to use

Why is JIF called a “proxy” rating? Although JIF is most appropriately applied to journals Although JIF is most appropriately applied to journals –By publishers determining sale prices –By editors comparing status –By authors looking for a prestigious journal to submit articles to It is often used to rate the quality of the articles appearing in a journal It is often used to rate the quality of the articles appearing in a journal –An article appearing in a journal with a high JIF is considered more important than an article in a low JIF journal The article receives its rating indirectly (by proxy) from the journal rating The article receives its rating indirectly (by proxy) from the journal rating

Examples of uses for accountability Faculty members Faculty members –Hiring decisions –Promotion decisions –Tenure decisions –Annual salary decisions Departmental funding Departmental funding –A factor in Great Britain universities Graduate students Graduate students –In China many scientific departments require publication in at least 2 JIF journals before a Ph. D. will be granted

How is the JIF computed? Conceptually Total Citations (2 preceding years Total Articles (2 preceding years =JIF Specific example Total Citations in 2006 to articles published in 2004 and 2005 Total Articles published in 2004 and 2005 =2006 JIF (Thomson/ISI JIF) Half of all JIFs are less than 1

Appearances can be deceiving The simple, scientific, and quantifiable JIF gets messier the closer you look The simple, scientific, and quantifiable JIF gets messier the closer you look There are concerns both about: There are concerns both about: –The mechanism used to compute the JIF –Some of the assumptions which underlie the JIF as a concept

What are these limitations? The limitations can be divided into 4 main areas The limitations can be divided into 4 main areas –Technical –Statistical –Philosophical (underlying assumptions) –Moral

Technical limitations The JIF applied to articles is a “proxy” rating The JIF applied to articles is a “proxy” rating –Even in some of the journals with the highest JIFs there have been bad articles (later withdrawn) Thomson only gathers citations from about 9,000 of 20,000-50,000 academic journals worldwide Thomson only gathers citations from about 9,000 of 20,000-50,000 academic journals worldwide The citations are only gathered during a two year period The citations are only gathered during a two year period There is a troubling lack of transparency (and possibly consistency) in how the citation and article counting is done There is a troubling lack of transparency (and possibly consistency) in how the citation and article counting is done –JIFs can’t be reliably replicated

Statistical limitations There is a possible problem with outliers There is a possible problem with outliers –In small populations an extreme instance can create a misleading effect Library journals in particular seldom publish more than 50 referred articles a year Library journals in particular seldom publish more than 50 referred articles a year –In such an environment just one or two very highly cited articles can establish a misleadingly high JIF There are statistical solutions to this problem but Thomson doesn’t use them There are statistical solutions to this problem but Thomson doesn’t use them

Philosophical (Assumptions) limitations While citations do a good job of measuring impact in traditional academic disciplines, that may not be the case in the professions While citations do a good job of measuring impact in traditional academic disciplines, that may not be the case in the professions In professions where the focus is on practical applications (librarianship, medicine, law), an article may have a large impact on behavior but generate few citations In professions where the focus is on practical applications (librarianship, medicine, law), an article may have a large impact on behavior but generate few citations In some disciplines downloads or other measures may be more useful in measuring impact In some disciplines downloads or other measures may be more useful in measuring impact –Reputational studies using expert opinion, for example, have very poor correlation with JIFs

Moral limitations As JIFs have become more important, there is a greater temptation to manipulate them unfairly As JIFs have become more important, there is a greater temptation to manipulate them unfairly –Like Claude Raines in Casablanca, librarians may be shocked to hear this –Review articles with their many citations can be used to increase a journal’s JIF –Sometimes publication acceptance depends on citing enough earlier articles published by a journal While such practices have been documented in other disciplines, so far library journals have been exempt! While such practices have been documented in other disciplines, so far library journals have been exempt!

So what does this all mean? As the song from Casablanca goes, “you must remember this…” As the song from Casablanca goes, “you must remember this…” JIFs can be a useful and helpful tool in determining the importance of a journal (and its articles)… JIFs can be a useful and helpful tool in determining the importance of a journal (and its articles)… But it must be used with caution and knowledge of its limitations But it must be used with caution and knowledge of its limitations

Follow-up reading for those interested (a short list) [put the web address of the bibliography here] [put the web address of the bibliography here]