Bibliometrics in support of research strategy & policy Anna Grey and Nicola Meenan

Overview Why use bibliometrics? Which indicators should we use? Case studies: Case study 1: University-level performance analyses Case study 2: Researcher-level performance analyses

Bibliometrics Academic literature reports on research Researchers cite previous studies Citations map knowledge transfer, proxy for research excellence Bibliometrics: a set of methods to quantitatively analyse academic literature Content analysis Citation analysis

Bibliometrics Bibliometrics is one part of a much wider picture of the research landscape Citations indicate that research is being ‘talked’ about, high citation counts correlate with research excellence Citations don’t tell you about context and circumstance Method? Review? Controversy?

Bibliometrics Bibliometrics are informative but are best used in support of other assessment methods (e.g. peer-review) and should not be used in isolation “Not everything that counts can be counted, and not everything that can be counted counts” William Bruce Cameron

Bibliometric indicators Bibliometric indicators can include: Output e.g. trends, publication types, journals used, research fields Citation impact e.g. average, trends, normalised citation impact, share of output in world’s top 10%/5%/1% Collaboration e.g. % international collaboration, geographical spread of collaboration

Factors to consider Time factors Type of publication Research field Level of aggregation (group, department, University) Individual factors (age, gender, career stage)

Normalised citation impact For any set of research: Citation impact= To normalise, we compare the citation count with average for the world, taking research field, year and publication type into account Normalised citation impact = World average citation impact = 1.0 [Sum of citations] [Count of publications] [Sum of citations * (World baseline)] [Count of publications]

Other useful research data Research income (total and per FTE) Research grant success rates REF results Funders by sector Market share Research student numbers and awards KT activity

Typical research office questions What are our strengths and weaknesses? What are the underlying trends? How does our group/department/university compare with others? Is this a good group/department/university to work with? If Professor X joined the University, would we improve? Do they bring new research specialisms?

Case study (1): University level performance

Case study (1): the University What research specialisms does the University of York have in Environmental Science? SciVal® database, Elsevier B.V., (downloaded on 04/06/14)

Case study (1): the University Has output in Environmental Science grown? SciVal® database, Elsevier B.V., (downloaded on 04/06/14)

Case study (1): the University How does York’s Environmental Science compare with other institutions? SciVal® database, Elsevier B.V., (downloaded on 04/06/14)

Case study (2): Researcher performance

Case study (2): the Researcher How frequently has Professor X published? SciVal® database, Elsevier B.V., (downloaded on 04/06/14)

Case study (2): the Researcher How frequently is Professor X being cited? Different ways of looking at how well-cited an individual is SciVal® database, Elsevier B.V., (downloaded on 04/06/14)

Case study (2): the Researcher In which journals does Professor X publish most frequently? SciVal® database, Elsevier B.V., (downloaded on 04/06/14)

Case study (2): the Researcher Which institutions does Professor X collaborate with most frequently? SciVal® database, Elsevier B.V., (downloaded on 04/06/14)

Case study (2): the Researcher In which research fields does Professor X publish most frequently? SciVal® database, Elsevier B.V., (downloaded on 04/06/14)

Case study (2): the Researcher How does Professor X compare with other researchers in their field? SciVal® database, Elsevier B.V., (downloaded on 04/06/14)

Closing remarks What decisions are you going to make based on the data? Use your data carefully! “With great power comes great responsibility” Be aware of where it’s sourced from Be aware of it’s limitations Always consider the context!