An overview of main bibliometric indicators: Dag W. Aksnes Data sources, methods and applications Nordic Institute for Studies in Innovation, Research and Education

Common indicators Number of publications used as an indirect measure of knowledge production The extent to which the publications have been cited in the subsequent scientific literature used as an indicator scientific impact and international visibility Co-authorship used as an indicator of collaboration, e.g. the extent of international collaboration

Common indicators in bibliometric studies of research IndicatorData Research outputPublications per unit Research profilePublications per field Scientific impactNumber of citations Research collaborationCo-authorship 3

Common data sources for bibliometric indicators Web of Science (traditionally) Scopus (increasingly) Various field specific databases (less often, field studies) Google scholar (increasingly, but controversial) CRIS-databases (new options) 4

Database requirements for bibliometric indicators (simplified) RequirementsWoSScopusGoogle Scholar CRIStinCRIS (prior) Adapted for bibliometric studies Complete indexing of all authors and author addresses ++-0, +- Citation counts+++-- Transparent and justified selection procedure of source material 0,+ -+- Coverage of natural sciences and medicine Coverage of engineering00++0 Coverage of A&H and SS--++0 Coverage of books and monographs

CRIS-data Norway Data on all types of scientific/scholarly publications in all fields of research are collected trough a documentation system (CRIStin). In the database the publication output is reported by the institutions as ordinary bibliographic references. A dynamic authority record of controlled scientific and scholarly publication channels ensures that references to non-scientific publications are not entered into the system. Publication data from professional bibliographic data sources (e.g. Web of Science) are imported to the documentation system in order to facilitate the registration of publications by the employees.

Database requirements for bibliometric indicators (simplified) RequirementsWoSScopusGoogle Scholar CRIStinCRIS (prior) Adapted for bibliometric studies Complete indexing of all authors and author addresses ++-0, +- Citation counts+++-- Transparent and justified selection procedure of source material 0,+ -+- Coverage of natural sciences and medicine Coverage of engineering00++0 Coverage of A&H and SS--++0 Coverage of books and monographs

Database requirements for bibliometric indicators (simplified) RequirementsWoSScopusGoogle Scholar CRIStinCRIS (prior) Adapted for bibliometric studies Complete indexing of all authors and author addresses ++-0, +- Citation counts+++-- Transparent and justified selection procedure of source material 0,+ -+- Coverage of natural sciences and medicine Coverage of engineering00++0 Coverage of A&H and SS--++0 Coverage of books and monographs

Publication types* in five major areas, Norwegian universities *) Weighted: 1 book = 5 articles (N=24,154)

Coverage of journal articles by Norwegian authors* 11 *) Accredited scientific and scholarly journals, CRIStin-data , Source: Gunnar Sivertsen (preliminary analysis)

Proportion of WoS indexed publications, selected SSH-departments, University of Bergen

Common indicators in bibliometric studies of research IndicatorData Research outputPublications per unit Research profilePublications per field Scientific impactNumber of citations Research collaborationCo-authorship 13

Addresses are the basis for indicators of: -Publication output per country or research unit

Total publication output 2012 (WoS) 15

National publication profile biology (Norway)

Counting of publications – methodological issues The question of how to handle papers involving co-authorship in general and international co-authorship in particular has been a recurrent issue in the literature on the use of bibliometric indicators Whole counting and fractional counting have been considered complementary methods According to Moed (2005), the integer or whole count method can be interpreted to measure participation, while the fractional counting method measures the number of papers creditable to a country. 19

Counting of publications – methodological issues Introduction of additional weighting systems, e.g. of first-authorship Weighting of books Publication types to be included 20

Publication channels - two levels Level 2- is giving extra weight includes only the leading and most selective international journals and publishers - 20 % of the total Level 1 80% of publications -”normal” wheight In the Norwegian system journals and publishers are divided into two levels

Funding model Norway: Weigted publication points based on publication level and publication channel: Level 1 Level 2 –Articles in scientific/scholarly journals: 1 3 –Articles in scientific/scholarly books: –Monographs: 5 8

Common indicators in bibliometric studies of research IndicatorData Research outputPublications per unit Research profilePublications per field Scientific impactNumber of citations Research collaborationCo-authorship 23

Journal titles are the basis for indicators of: - Allocation of publications to fields of research - Research profiles - Field normalization

Number of articles* by field, *) Indexed in Web of Science 25

Relative Specialization Index for ‘traditional’ universities (blue line equals the world average) ( ) UiB UiO 26 NTNU UMB

Common indicators in bibliometric studies of research IndicatorData Research outputPublications per unit Research profilePublications per field Scientific impactNumber of citations Research collaborationCo-authorship 27

Citations are the basis for indicators of: - Impact (field normalized)

Citation rates by field (2005, 5 year-window) DisciplineCit/pubUncitedDisciplineCit/pubUncited Molecular Biology & Genetics22.44 %Geosciences8.812 % Immunology20.73 %Physics8.417 % Microbiology15.64 % Plant & Animal Science6.916 % Biology & Biochemistry15.17 %Agricultural Sciences6.916 % Space Science %Economics-Business5.620 % Clinical Medicine % Social Sciences, general5.022 % Chemistry %Engineering4.626 % Environment/Ecology10.39 %Mathematics3.329 % Psychiatry/Psychology %Computer Science2.948 %

Number of citations per year per article

Cumulative share of citations and publications, % ( years cit. win)

Citations counts – different sources ARTL_TITLEJRNLFULLVOLYEAR Citations - Web of ScienceCitations - Scopus Citations - Google scholar A macro study of self-citationSCIENTOMETRICS Characteristics of highly cited papers RESEARCH EVALUATION Peer reviews and bibliometric indicators: a comparative study at a Norwegian university RESEARCH EVALUATION Does self-citation pay?SCIENTOMETRICS Scientific productivity and group size: A bibliometric analysis of Norwegian microbiological researchSCIENTOMETRICS The effect of highly cited papers on national citation indicatorsSCIENTOMETRICS Citation rates and perceptions of scientific contributionJASIST Validation of bibliometric indicators in the field of microbiology: A Norwegian case studySCIENTOMETRICS Researchers' perceptions of citationsRESEARCH POLICY When different persons have an identical author name. How frequent are homonyms?JASIST The Structure and Development of Polar Research ( ): a Publication-Based Approach ARCTIC ANTARCTIC AND ALPINE RESEARCH Ranking national research systems by citation indicators. A comparative analysis using whole and fractionalised counting methods JOURNAL OF INFORMETRICS Are Female Researchers Less Cited? A Large-Scale Study of Norwegian Scientists JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE AND TECHNOLOGY Science policy and the driving forces behind the internationalisation of science: The case of Norway Science and Public Policy Total

Google scholar vs Web of science

Relative citation index Relative citation index field = (Observed Citation Rate/World Citation Average of Subfield)X100 Relative citation index journal = (Observed Citation Rate/Average citation rate of journal)X100 Example (shown for one article only): –A regular journal article in Acta Chrystallographica E, 2005 has been cited 3 times. –The articles published in Acta Chrystallographica E were in contrast cited 1.74 times on average this year. –The Relative citation index – journal is: (3/1.74)*100 = 172. –The world-average citation rate for the subfield which this journal is assigned to is 3.7 for articles published this year. The Relative citation index – field is: (3/3.7)*100 = 81

Field normalized relative citation indexes,

Field normalized relative citation indexes of universities and university hospitals ( ) across subject fields 37

H-index Articles ranked by decreasing citation numbers. A scientist with an index of h has published h papers each of which has been cited by at least h times –can be manually determined using citation databases or using automatic tools –attempts to measure both the scientific productivity and the apparent scientific impact of a scientist –Reduces the effect of highly cited papers –But not without shortcomings: No field normalisation. H [read: age] index.

Journal impact factor One of the most well-known bibliometric indicators Originally introduced by Eugene Garfield as a measure of the frequency with which the average article in a journal has been cited Average citation rate for 1 and 2 years old articles published in a particular journal: Number of citations received in year T to the articles published in a journal during the years T-1 og T-2 divided by the number of articles published in the journal during the years T-1 og T-2

Common indicators in bibliometric studies of research IndicatorData Research outputPublications per unit Research profilePublications per field Scientific impactNumber of citations Research collaborationCo-authorship 40

Addresses are the basis for indicators of: -Publication activity per university -Research collaboration (combinations of addresses in publications)

Map of scientific collaborations 2005 to 2009

Norwegian articles with and without international co- authorship,

The institutional national and international collaboration profile of the University of Bergen based on co-authorship

The international collaboration profile of the University of Bergen based on co-authorship

The institutional international collaboration profile of the University of Bergen based on co-authorship

Evaluation of research performance - an important application of bibliometrics Growing, often controversial, policy interest to use bibliometric techniques as output measures of research productivity and impact Bibliometrics evolved to a standard tool in science policy and research evaluation 47

Science policy applications Monitoring research systems Evaluations Funding

Applications in research evaluations Levels of analyses –Country –Institutions –Departments –Divisions/research groups –(Individuals – anonymous presentation)

Bibliometric indicators for funding purposes In 2004, Norway implemented a bibliometric model for performance based budgeting of research institutions –The funding of these institutions is now partially based on the measurement of their scientific and scholarly publishing –National bibliographic database (CRIStin) used for this purpose –Similar system for research institutes and hospitals

Questions – comments?