Introduction to Citation Metrics Elizabeth Lee San Bao | elizabeth.lee@nus.edu.sg 2 March 2016

Session Outline What are Citation Metrics? Why do we use Citation Metrics? Types of Citation Metrics Limitations of Citation Metrics Managing your Research Publication History

What are Citation Metrics?

What are Citation Metrics? Metrics used to count, measure, rank, analyse and compare a journal / article / author / institution / organization’s research impact, based on their scholarly published articles and their citations One of the most widely used methods in the field of bibliometrics – the statistical analysis of written publications such as books and articles. Citation metrics base their data from citation indexes, which are produced by different publishers, e.g. Thomson Reuters, Elsevier

What are Citation Metrics? We will cover the following citation metrics: Journal Metrics Journal Impact Factor 5-Year Journal Impact Factor Journal Immediacy Index Author Metrics Number of publications Citation Count H-index

Why do we use Citation Metrics?

Why do we use Citation Metrics? Track impact of a researcher’s scholarly publications Record achievement for academic promotion and tenure, research funding or job applications Compare quality of scholarly work with other researchers / institutions Used by University Faculty / Department / Research Institutes for reporting, benchmarking and comparison of staff  annual staff performance assessment Build a research profile so others can find and follow one's work. For researchers / institutions: Assumption: Researchers with high citation counts are writing articles that describe important and influential research

Why do we use Citation Metrics? Be aware of which are the leading journals in specific subject disciplines and to read up on them Be alert of hot research topics in their subject disciplines Examine which top peer-reviewed scholarly articles to read Trace historical development of certain research topics and prominent scientists associated with them Identify any research gaps that may lead to potential dissertation topics For graduate students:

Types of Citation Metrics Journal Metrics Author Metrics

Journal Metric 1: Journal Impact Factor Evaluate or compare a journal's relative importance to others in the same field Average number of times articles from a journal published in the past two years have been cited in the JCR year Based on data from the Incites Journal Citation Reports (JCR) by Thomson Reuters – a quantitative tool for ranking, evaluating, categorizing and comparing journals JCR Year - The year of the JCR edition displayed ارزیابی و مقایسه اهمیت نسبی یک مجله نسبت به مجلات همان حوزه. * For more information, you can also visit the NUS LibGuide on NUS Cited Reference Searching.

Journal Metric 1: Journal Impact Factor Journal Impact Factor is defined as the average number of times articles from the journal published in the past two years have been cited in the JCR year. For the journal “Reviews of Modern Physics”, the Journal Impact Factor is shown below: * You can refer to our LibFAQ on Journal Impact Factor here.

Journal Metric 2: 5-Year Journal Impact Factor Average number of times articles from the journal published in the past five years have been cited in the JCR year Calculated by dividing the number of citations in the JCR year by the total number of articles published in the five previous years Used to better gauge the impact of journals in fields where the INFLUENCE OF PUBLISHED RESEARCH EVOLVES OVER A LONGER PERIOD OF TIME میانگین تعداد دفعاتی که مقالات 5 سال گذشته مجله در سال مدنظر مورد ارجاع قرار گرفته اند.. ضریب نفوذ مقالات چاپ شده در بازه بزرگتری از زمان را می سنجد.

Journal Metric 2: 5-Year Journal Impact Factor For the journal “Reviews of Modern Physics”, the 5-Year Journal Impact Factor is shown:

Journal Metric 3: Journal Immediacy Index Average number of times an article is cited in the year it is published. Indicates how quickly articles in a journal are cited Form of evaluation to see which journals publish cutting-edge research. میانگین تعداد دفعاتی که یک مقاله در همان سالی که نشر شده مورد ارجاع هم قرار گرفته است. Because it is a per-article average, the Immediacy Index tends to discount the advantage of large journals over small ones. However, frequently issued journals may have an advantage because an article published early in the year has a better chance of being cited than one published later in the year. Many publications that publish infrequently or late in the year have low Immediacy Indexes. For comparing journals specializing in cutting-edge research, the immediacy index can provide a useful perspective.

Journal Metric 3: Journal Immediacy Index For the journal “Reviews of Modern Physics”, the Journal Immediacy Index is shown:

Other Journal Metrics Eigenfactor’s algorithm ranks journals similarly to how Google ranks websites. It assigns a greater weight to those citations coming from influential journals and does not count journal self-citations. Eigenfactor Score The Article Influence Score measures the relative importance of the journal on a per-article basis. It is calculated by dividing the Eigenfactor score by the number of articles published in the journal. Article Influence Score Uses algorithms similar to Google PageRank and is based on the prestige of the citing journal. With SJR, the subject field, quality and reputation of the journal has a direct effect on the value of a citation. Normalizes for differences in citation behaviour between subject fields. SCImago Journal Rank (SJR) The calculation of the SJR indicator is very similar to the Eigenfactor score, with the former being based on the Scopus database and the latter on the Web of Science database.

Impact per Paper (SNIP) Google Scholar Metrics Other Journal Metrics Developed by Leiden University’s Centre for Science and Technology Studies (CWTS), SNIP measures contextual citation impact by weighting citations based on the total number of citations in a subject field. If there are fewer total citations in a research field, then citations are worth more in that field, and vice versa. Source Normalised Impact per Paper (SNIP) Analyses articles published in the last 5 calendar years and its metrics are based on citations from all articles that are indexed in Google Scholar. Uses h5-index and h5-median as its journal metrics, where h is the largest number of articles that have each been cited h times. Google Scholar Metrics

Author Metrics 1 & 2: Number of Publications & Citation Count Measuring the total number of publications (usually, journal articles) during author’s academic career Calculating the total number of citations to an author’s published articles (Done by author search in Web of Science)

Explaining Citation Count (Citation Analysis) 2005 1998 2007 2003 Paper A’s Cited References 2012 2013 Paper A’s Citing Articles Paper A 2010 Number of citing articles for Paper A = Citation Count for Paper A “X papers have cited my Paper A as reference.”

Author Metrics 1 & 2: Number of Publications & Citation Count In different subject disciplines, the rate of publishing books and journal articles will vary.  do these metrics above reflect true quality of the research work done? What about considering the following? : Being positively assessed during the peer-review process Measures of prestige such as invitations / honorary membership on editorial boards, funding grants and industrial awards

Author Metric 3: h-index Developed in 2005 by Jorge Eduardo (J.E.) Hirsch in his paper “An index to quantify an individual's scientific research output” He states that: “A scientist has index h if h of his/her Np papers have at least h citations each, and the other (Np − h) papers have no more than h citations each.” In other words, an author with an index of h has published h papers, each of which has been cited in other papers at least h times.

Limitations of Citation Metrics

Limitations of Citation Metrics When conducting analysis with Citation Metrics, always consider the Citation Sources. Citation Sources are not perfect. Coverage of indexed publications (Citation Sources) Type (Journals, Reports, Books, Conference Proceedings, etc.) Volume (Large vs small number of indexed sources) Time Period (Established journals with longer publication history) Geographical Range (International vs Regional coverage) Language (i.e. bias towards English Language) Subject (Specific vs Multidisciplinary sources) Human Errors in Indexing Typographical Transcription

Limitations of Citation Count Citation Count alone does not determine the: Full range of a Researcher’s activity and performance Whether citations are due to a longer publication history Researcher’s currency of research and recent performance Standing or prestige of the journals the articles are published in or cited by If citations are viewed positively or negatively Ranking of researchers from different disciplines Making Research Count: Analyzing Canadian Academic Publishing Cultures, Paul Jarvey, Alex Usher, Lori McElroy, Higher Education Strategy Associates, June 2012

Putting Citation Count in Perspective As at 27 Jan 2016 The Most Highly Cited Paper in Publishing History: Protein Determination by Oliver H. Lowry, http://www.jbc.org/content/280/28/e25.full

Putting Citation Count in Perspective “…it is flattering to be the ‘most cited author’ but I am afraid it does not signify great scientific accomplishment… Although method development is usually a pretty pedestrian affair, others doing more creative work have to use methods and feel constrained to give credit for the same…” Oliver Howe Lowry Lowry OH,  Rosebrough NJ, Farr AL & Randall RJ.  Protein measurement with the Folin phenol reagent. J. Biol.  Chem. 193: 265-75, 1951. Classic Paper.  Citation Classic Commentary http://garfield.library.upenn.edu/classics1977/A1977DM02300001.pdf

Managing your Research Publication History

Managing your Research Publication History Be consistent in the use of your name syntax in all your publications, e.g. Stephen Lee Jin Cheng Avoid author disambiguation by choosing a distinct name that will ease differentiation from other authors. List your institutional affiliations over the course of your career correctly, including: Name of Institution - National University of Singapore Faculty - Faculty of Science instead of Science Faculty, Sci Fac, etc Department - Department of Physics instead of Phy Dept For Research Group - Include the Faculty (if applicable) and Department Lee, Stephen Jin Cheng Lee, J.C. Stephen Lee, SJC

Managing your Research Publication History Check that your publications are correctly listed and indexed in major databases such as Web of Science and Scopus Promote your research online profile and research publications by participating in researcher identification registries such as: ORCID (Open Researcher and Contributor ID) ResearcherID Google Scholar Citations

Managing your Research Publication History

List of References Library Guides: http://libguides.nus.edu.sg/citedref | http://guides.library.jhu.edu/metrics Related Library FAQ: How do I get impact factors for journals? Eigenfactor Score / Article Influence Score | SCImago Journal Rank (SJR) | Source Normalised Impact per Paper (SNIP) | Google Scholar Metrics ORCID (Open Researcher and Contributor ID) | ResearcherID | Google Scholar Citations Professor Andrew Wee Thye Shen’s NUS Page: http://www.physics.nus.edu.sg/~phyweets/pub.html “What does it mean to be #2 in Impact?”, Meredith McVeigh “An index to quantify an individual's scientific research output”, Jorge Eduardo (J.E.) Hirsch “Making Research Count: Analyzing Canadian Academic Publishing Cultures”, Paul Jarvey, Alex Usher, Lori McElroy, Higher Education Strategy Associates “The Most Highly Cited Paper in Publishing History: Protein Determination by Oliver H. Lowry”, Nicole Kresge, Robert D. Simoni and Robert L. Hill Protein measurement with the Folin phenol reagent. J. Biol.  Chem. 193: 265-75, 1951. Classic Paper.  Citation Classic Commentary. Lowry OH,  Rosebrough NJ, Farr AL & Randall RJ. 

More Useful References NUS Library Books: Beyond bibliometrics : harnessing multidimensional indicators of scholarly impact / edited by Blaise Cronin and Cassidy R. Sugimoto The publish or perish book : your guide to effective and responsible citation analysis / Anne-Wil Harzing Bibliometrics and citation analysis : from the Science citation index to cybermetrics / Nicola De Bellis Article Level Metrics, PLOS SPARC Article-Level Metrics Primer, SPARC®, the Scholarly Publishing and Academic Resources Coalition Open Access Explained! , Piled Higher and Deeper (PHD Comics) Standing up for Science 3: Peer Review - The nuts and bolts (A guide for early career researchers), Sense About Science Science Metrics, Nature

Your (REAL) Impact Factor (from PHD Comics) First published on 8 Dec 2008, http://www.phdcomics.com/comics/archive.php?comicid=1108

Thank you for attending today Thank you for attending today! Please fill in the feedback form: http://bit.ly/1PRuptU Elizabeth Lee elizabeth.lee@nus.edu.sg Science Library Information Desk sclib@nus.edu.sg | Tel: 6516 2454