WISER: Bibliometrics II The Black Art of Citation Rankings Angela Carritt Juliet Ralph November 2011 These slides are available on /wiser/presentations

Overview of Session What are bibliometrics? Why bother? Problems Bibliometric measures for …a researcher …an institution …a country …a journal using Web of Science, Scopus, and other analytical tools

What are bibliometrics? Citation analysis

Why bother Benchmarking of departments and research groups Grant applications Recruitment of individuals Where to publish

Bibliometrics & the REF Citation data will not be as important as initially anticipated Some panels will use citation data Panel A – Life Sciences(1) Some parts of Panel B - Earth and environmental sciences, chemistry, physics and computer science(1) Some parts of panel C – some areas of geography, environmental studies and archaeology but not all, economics/econometrics(1) Expert review will be the primary way of measuring impact even where citation data is used REF team will provide citation data (from Scopus) More information - HEFCE, Ref Assessment framework and guidance on submissions ( (1) Information from Gibney, Elizabeth ‘REF panels to differ on impact and citation use’

The h-index: to quantify an individual’s research output Aims to measure productivity and impact. h-index = number of papers you have published which have that number of citations or more. e.g. an h-index of 5 means you have 5 papers which have been cited at least 5 times each.

Your h-index in Web of Science Do an author search. Get list of papers by them. Sort by Times Cited. Click on Create Citation Report - to analyse batch of papers.

Calculation of h-index Analysing 208 articles in WoS. H-index = 53

53 articles cited 53 times or more

Citation tracking & analysis in SCOPUS Scopus covers 18,000 journals & conference proceedings Science, Medicine, Social Sciences & Humanities Each record for a paper shows the number of times it has been cited in Scopus since 1996 Similar analytical tools to Web of Science

Your h-index in Scopus Do an author search. Tick author name. Click on View Citation Overview.

h-index = 21. Why? Based on papers in Scopus published after 1995.

Lower if exclude self-citations h-index = 17

Your h-index and Google Scholar

Calculate it with Publish or Perish Free download. Uses citation data from Google Scholar.

Bibliometrics for an institution In Scopus, Affiliation search allows search by institution name - university of oxford retrieves same results as oxford university

Overview of institution

Publications by a department Search Address field. Can search by postcode or name of department/college. NB. Some authors use dept postcode, some use main University one (OX1 2JD).

Re Research Committee Recommendation (1) All authors should use ‘Oxford University’ or ‘University of Oxford’ in their publication address, to ensure that the publication may be captured by citation databases. (2) [may also] cite either dept or college.

Oxford & the REF

Symplectic Elements Oxford’s tool for gathering references and submission for the REF. Automatic searching of databases such as Web of Science & Scopus. Is your department using it? For more information go to Contact

Bibliometrics for journals Bibliometrics can be used to measure the influence of academic journals Help you to decide where to publish

Bibliometrics & journals: Tools ISI Journal Citation Reports (JCR) (ISI Web of Science) SCImago Journal Ranking (SJR) and SNIP (Elsevier Scopus) Eigenfactor (sponsored by the Bergstrom Lab in the Department of Biology at the University of Washington

Journal Citation Reports (JCR) Based on citation data from Web of Science Covers > 5,900 journals in science and technology > 1,700 journals in the social sciences

JCR on Web of Knowledge

JCR Impact Factor - Number of times the “average” article published in the previous 2 (or 5) years was cited this year. Calculated: no. of citations to articles published in the last 2 (or 5) years ÷ no. of articles published in same period. Immediacy Index Measures how quickly articles are cited. Calculated: no. of citations to articles published this year ÷ no. of articles published this year. Cited Half-Life - How many years you have to go back to account for 50% of citations to the journal. e.g. 50% of citations were to articles published in the last 3.5 years. The rest cited earlier articles.

Detailed view

Detailed view continued

Citations TO the journal by year of cited article (e.g. 333 of this year’s citations to Biological Review were to articles published in in 2005 ) Detailed view continued

Citations from Biological Review (to other journals and self cites) by year of cited article E.g. 334 citations from Biological Reviews journal cited articles published in 2007 Detailed view continued

Type of articles included

Eigenfactor Metrics Take into account prestige of citing sources Use “Google style” algorithms Attempts to measure how often the average researcher would encounter the journal 2 metrics Eigenfactor – increases with the size of the journal Article Influence – Takes into account number of articles published. More comparable to the JCR impact factor Google’s PageRank from

Eigenfactor

SJR – an alternative impact factor SCImago Journal Rank developed by Elsevier in partnership with Spanish academics. Citation data from Scopus 50% more journals than ISI Web of Science. Weights citations according to the status of the citing journal Updated every 2 months More info at in-detail/tools/journalanalyzer/ in-detail/tools/journalanalyzer/

SCImago and the SJR

SCImago Journal Ranking

SCImago journal search More metrics Orange line = average number of citations (i.e. same as JCR) Purple line = SJR

Comparing journals

Measuring impact for a country

SNIP – also by Elsevier Source Normalized Impact per Paper. Weights citation counts according to the total number of citations in the subject area SNIP aims to account for differences in citation potential and topicality across research fields. ‘Citation potential’ (citation frequency) higher in life sciences than maths, engineering, social science. Higher in basic science than applied or clinical journals. Scopus is again the data source.

SNIP (and SJR)

SJR Choose journals here Use slider to select different time periods Use tabs to see different metrics

SJR, Snip… SJR – takes into account prestige of citing journal SNIP – contextual impact. Weights citations according to total number of citations in the subject Total number of citations Number of articles published % of articles not cited at all % of review articles

Favourably reviewed in NatureNature SJR & SNIP freely available – not dependent on subscriptions to Scopus. Worthy challenger to ISI.

Journal Impact Factors: Problems Use with caution…Results are skewed by many factors… Size Frequency / time of publication Type of content - review articles are more heavily cited than original research… Journals that are not indexed by WOS / Scopus are disadvantaged Non English Language journals disadvantaged Problems when journals change names Results are not comparable across discipline (some journals in the wrong discipline) Journal impact factors should NEVER be used to assess impact of researchers / groups etc

Other ranking tools ScienceWatch Weekly tracking of highly-cited papers and topics Free Web resource for science metrics and analysis. Includes interviews, essays, podcasts and profiles from scientists, journals, institutions, and nations. Uses citation data from Thomson Reuters (owner of ISI Web of Science).

Essential Science Indicators NEW! Oxford now has subscription to ESI. Rankings for Scientists, Institutions, Countries, Journals, Papers.

Help & info See our Bibliometrics guide at bibliometrics bibliometrics

Brief bibliography General Broadus, R. N., “Towards a definition of Bibliometrics” Scientometrics, vol. 12, nos 5-6, (1987) HEFCE papers, reports, papers and pilots on the use of bibliometrics in the H-Index Hirsch, J. E. (15 November 2005). "An index to quantify an individual's scientific research output". PNAS 102 (46): Ball, P. “Index aims for fair ranking of scientists”, Nature 2005 Aug : 900

Brief bibliography Journal Impact Factors and the JCR The Thomson Reuters Impact Factor (originally published in the Current Contents print editions June 20, Garfield, E. "The agony and the ecstasy: the history and meaning of the Journal Impact Factor“ Paper at the International Congress on Peer Review And Biomedical Publication, Chicago, September 16, Want more… Wikipedia entries on the following topics include useful bibliographies: the h-index, journal impact factors and the Eigenfactor A Google Scholar ( search will return many useful articles including subject studies on the use of bibliometrics