Introduction to Citation Metrics Elizabeth Lee San Bao | 5 March 2015

Session Outline  What are Citation Metrics?  Why do we use Citation Metrics?  Types of Citation Metrics  Limitations of Citation Metrics  Managing your Research Publication History

What are Citation Metrics?

 Metrics used to count, measure, rank, analyse and compare a journal / article / author / institution / organization’s research impact, based on their scholarly published articles and their citations  One of the most widely used methods in the field of bibliometrics – the statistical analysis of written publications such as books and articles.  Citation metrics base their data from citation indexes, which are produced by different publishers, e.g. Thomson Reuters, Elsevier  We will cover the following citation metrics today: 1.Journal Metrics 2.Author Metrics

Explaining Citation Count (Citation Analysis) Paper A Paper A’s Cited References Paper A’s Citing Articles Number of citing articles for Paper A = Citation Count for Paper A “X papers have cited my Paper A as reference.”

Why do we use Citation Metrics?

For researchers / institutions:  Used to track impact of a researcher’s scholarly publications  Record achievement for the purpose of academic promotion and tenure, research funding or job applications  Used as a way to compare quality of scholarly work with other researchers / institutions  Used by University Faculty / Department / Research Institutes for reporting, benchmarking and comparison of staff  annual staff performance assessment  Build a research profile so others can find and follow one's work. Assumption: Researchers with high citation counts are writing articles that describe important and influential research

Why do we use Citation Metrics? For graduate students:  To be aware of which are the leading journals in specific subject disciplines and to read up on them  To be alert of hot research topics in their subject disciplines  To examine which top peer-reviewed scholarly articles to read  To trace historical development of certain research topics and prominent scientists associated with them  To identify any research gaps that may lead to potential dissertation topics

Types of Citation Metrics a)Journal Metrics b)Author Metrics

Journal Metric 1: Journal Impact Factor  Used in evaluating or comparing a journal's relative importance to others in the same field.  Represents the frequency with which an average article from a journal is cited in a particular year.  Calculated based on data from the Journal Citation Reports (JCR) by Thomson Reuters – a quantitative tool for ranking, evaluating, categorizing and comparing journals.Journal Citation Reports (JCR) * For more information, you can also visit the NUS LibGuide on NUS Cited Reference Searching.NUS LibGuide on NUS Cited Reference Searching

Journal Metric 1: Journal Impact Factor  Journal Impact Factor is defined as the average number of times articles from the journal published in the past two years have been cited in the JCR year.  For the journal “Reviews of Modern Physics”, the Journal Impact Factor is shown below: * You can refer to our LibFAQ on Journal Impact Factor here.here

Journal Metric 1: Journal Impact Factor

Activity: Journal Impact Factor Launch Journal Citation Reports here.here  Find the impact factor of the journal, Acta Crystallographica Section A  Look for the Impact Factor Trend.  What do you notice? 12 Source: What does it mean to be #2 in Impact?What does it mean to be #2 in Impact?  Select “View a group of journals by: Subject category”  Physics, Applied  View journals by Journal Impact Factor.  View journals from other subjects: a)Mathematics b)Engineering c)Chemistry  What do you notice?

Journal Metric 2: 5-Year Journal Impact Factor  Average number of times articles from the journal published in the past five years have been cited in the JCR year.  It is calculated by dividing the number of citations in the JCR year by the total number of articles published in the five previous years.  This metric can be used to better gauge the impact of journals in fields where the influence of published research evolves over a longer period of time.  For the journal “Reviews of Modern Physics”, the 5-Year Journal Impact Factor is shown:

Journal Metric 3: Journal Immediacy Index  Average number of times an article is cited in the year it is published.  Indicates how quickly articles in a journal are cited  form of evaluation to see which journals publish cutting-edge research.  For the journal “Reviews of Modern Physics”, the Journal Immediacy Index is shown:

Other Journal Metrics Journal MetricDescription Eigenfactor Score / Article Influence ScoreEigenfactor’s algorithm ranks journals similarly to how Google ranks websites. It assigns a greater weight to those citations coming from influential journals and does not count journal self-citations. The Article Influence Score measures the relative importance of the journal on a per-article basis. It is calculated by dividing the Eigenfactor score by the number of articles published in the journal. SCImago Journal Rank (SJR)Uses algorithms similar to Google PageRank and is based on the prestige of the citing journal. With SJR, the subject field, quality and reputation of the journal has a direct effect on the value of a citation. Normalizes for differences in citation behaviour between subject fields. Source Normalised Impact per Paper (SNIP) Developed by Leiden University’s Centre for Science and Technology Studies (CWTS), SNIP measures contextual citation impact by weighting citations based on the total number of citations in a subject field. If there are fewer total citations in a research field, then citations are worth more in that field, and vice versa. Google Scholar MetricsAnalyses articles published in the last 5 calendar years and its metrics are based on citations from all articles that are indexed in Google Scholar. Uses h5-index and h5-median as its journal metrics, where h is the largest number of articles that have each been cited h times.

Author Metric 1: Number of Publications & Citation Count  Measuring the total number of publications (usually, journal articles) during author’s academic career  Calculating the total number of citations to an author’s published articles (Done by author search in Web of Science)  In different subject disciplines, the rate of publishing books and journal articles will vary.  do these numbers above reflect true quality of the research work done? What about considering the following? :  Being positively assessed during the peer-review process  Measures of prestige such as invitations / honorary membership on editorial boards, funding grants and industrial awards

Author Metric 1: Citation Count – Author Search in WOS Name Variants: a)Wee Thye Shen, Andrew b)Andrew Wee T S c)Wee, Andrew Thye Shen d)Wee ATS e)Wee A f)Wee TSA Example: Professor Andrew Wee Publication list updated as of 27 Nov 2014 * You can view Prof Wee’s ResearcherID Profile Page Prof Wee’s ResearcherID Profile Page for more information.

Author Metric 1: Citation Count – Author Search in WOS

Author Metric 2: h-index  Developed in 2005 by Jorge Eduardo (J.E.) Hirsch in his paper “An index to quantify an individual's scientific research output”“An index to quantify an individual's scientific research output”  He states that: “A scientist has index h if h of his/her Np papers have at least h citations each, and the other (Np − h) papers have no more than h citations each.”  In other words, an author with an index of h has published h papers each of which has been cited in other papers at least h times.

Author Metric 2: h-index “Obviously, a single number can never give more than a rough approximation to an individual's multifaceted profile, and many other factors should be considered in combination in evaluating an individual.” “There is considerable variation in the skewness of citation distributions even within a given subfield, and for an author with a relatively low h that has a few seminal papers with extraordinarily high citation counts, the h index will not fully reflect that scientist's accomplishments.” “Conversely, a scientist with a high h achieved mostly through papers with many coauthors would be treated overly kindly by his or her h.” Hirsch, J.E. An index to quantify an individual's scientific research output, PNAS, vol. 102 no. 46 (2005) Jorge E. Hirsch

Limitations of Citation Metrics

When conducting analysis with Citation Metrics, always consider the Citation Sources. Citation Sources are not perfect.  Coverage of indexed publications (Citation Sources) a)Type (Journals, Reports, Books, Conference Proceedings, etc.) b)Volume (Large vs small number of indexed sources) c)Time Period (Established journals with longer publication history) d)Geographical Range (International vs Regional coverage) e)Language (i.e. bias towards English Language) f)Subject (Specific vs Multidisciplinary sources)  Errors in Indexing a)Typographical b)Transcription

Limitations of Citation Count Citation Count alone does not determine the:  Full range of a Researcher’s activity and performance  Whether citations are due to a longer publication history  Researcher’s currency of research and recent performance  Standing or prestige of the journals the articles are published in or cited by  If citations are viewed positively or negatively  Ranking of researchers from different disciplines Making Research Count: Analyzing Canadian Academic Publishing CulturesMaking Research Count: Analyzing Canadian Academic Publishing Cultures, Paul Jarvey, Alex Usher, Lori McElroy, Higher Education Strategy Associates, June 2012

Putting Citation Count in Perspective As at 11 Feb 2015 The Most Highly Cited Paper in Publishing History: Protein Determination by Oliver H. Lowry,

Putting Citation Count in Perspective “…it is flattering to be the ‘most cited author’ but I am afraid it does not signify great scientific accomplishment… Although method development is usually a pretty pedestrian affair, others doing more creative work have to use methods and feel constrained to give credit for the same…” Lowry OH, Rosebrough NJ, Farr AL & Randall RJ. Protein measurement with the Folin phenol reagent. J. Biol. Chem. 193: , Classic Paper. Citation Classic Commentary Oliver Howe Lowry

Managing your Research Publication History

 Be consistent in the use of your name syntax in all your publications, e.g. Stephen Lee Jin Cheng  Avoid author disambiguation by choosing a distinct name that will ease differentiation from other authors.  List your institutional affiliations over the course of your career correctly, including: a)Name of Institution - National University of Singapore b)Faculty - Faculty of Science instead of Science Faculty, Sci Fac, etc c)Department - Department of Physics instead of Phy Dept d)For Research Group - Include the Faculty (if applicable) and Department Lee, Stephen Jin ChengLee, J.C. StephenLee, SJC

Managing your Research Publication History  Check that your publications are correctly listed and indexed in major databases such as Web of Science and Scopus  Promote your research online profile and research publications by participating in researcher identification registries such as: a)ORCID (Open Researcher and Contributor ID)ORCID (Open Researcher and Contributor ID) b)ResearcherIDResearcherID c)Google Scholar CitationsGoogle Scholar Citations

Managing your Research Publication History

List of References  Library Guides: |  Related Library FAQ: How do I get impact factors for journals? | LibFAQ on Citation AnalysisHow do I get impact factors for journals?LibFAQ on Citation Analysis  Eigenfactor Score / Article Influence Score | SCImago Journal Rank (SJR) | Source Normalised Impact per Paper (SNIP) | Google Scholar Metrics Eigenfactor Score / Article Influence ScoreSCImago Journal Rank (SJR)Source Normalised Impact per Paper (SNIP)Google Scholar Metrics  ORCID (Open Researcher and Contributor ID) | ResearcherID | Google Scholar Citations ORCID (Open Researcher and Contributor ID)ResearcherIDGoogle Scholar Citations  Professor Andrew Wee Thye Shen’s NUS Page:  “What does it mean to be #2 in Impact?”, Meredith McVeigh “What does it mean to be #2 in Impact?  “An index to quantify an individual's scientific research output”, Jorge Eduardo (J.E.) Hirsch “An index to quantify an individual's scientific research output”  “Making Research Count: Analyzing Canadian Academic Publishing Cultures”, Paul Jarvey, Alex Usher, Lori McElroy, Higher Education Strategy Associates “Making Research Count: Analyzing Canadian Academic Publishing Cultures  “The Most Highly Cited Paper in Publishing History: Protein Determination by Oliver H. Lowry”, Nicole Kresge, Robert D. Simoni and Robert L. Hill “The Most Highly Cited Paper in Publishing History: Protein Determination by Oliver H. Lowry  Protein measurement with the Folin phenol reagent. J. Biol. Chem. 193: , Classic Paper. Citation Classic Commentary. Lowry OH, Rosebrough NJ, Farr AL & Randall RJ. Protein measurement with the Folin phenol reagent. J. Biol. Chem. 193: , Classic Paper. Citation Classic Commentary

More Useful References  NUS Library Books: a)Beyond bibliometrics : harnessing multidimensional indicators of scholarly impact / edited by Blaise Cronin and Cassidy R. SugimotoBeyond bibliometrics : harnessing multidimensional indicators of scholarly impact / edited by Blaise Cronin and Cassidy R. Sugimoto b)The publish or perish book : your guide to effective and responsible citation analysis / Anne-Wil HarzingThe publish or perish book : your guide to effective and responsible citation analysis / Anne-Wil Harzing c)Bibliometrics and citation analysis : from the Science citation index to cybermetrics / Nicola De BellisBibliometrics and citation analysis : from the Science citation index to cybermetrics / Nicola De Bellis  Article Level Metrics, PLOS Article Level Metrics  SPARC Article-Level Metrics Primer, SPARC®, the Scholarly Publishing and Academic Resources Coalition SPARC Article-Level Metrics Primer  Open Access Explained!, Piled Higher and Deeper (PHD Comics) Open Access Explained!Piled Higher and Deeper (PHD Comics)  Standing up for Science 3: Peer Review - The nuts and bolts (A guide for early career researchers), Sense About Science Standing up for Science 3: Peer Review - The nuts and bolts (A guide for early career researchers)  Science Metrics, Nature Science Metrics

Your (REAL) Impact Factor (from PHD Comics) First published on 8 Dec 2008,

Thank you for attending today! Please fill in the feedback form: Elizabeth Lee Science Library Information Desk | Tel: