1. An overview of main bibliometric indicators: Dag W. Aksnes Data sources, methods and applications Nordic Institute for Studies in Innovation, Research and Education

2. Common indicators Number of publications used as an indirect measure of knowledge production The extent to which the publications have been cited in the subsequent scientific literature used as an indicator scientific impact and international visibility Co-authorship used as an indicator of collaboration, e.g. the extent of international collaboration

3. Common indicators in bibliometric studies of research IndicatorData Research outputPublications per unit Research profilePublications per field Scientific impactNumber of citations Research collaborationCo-authorship 3

4. Common data sources for bibliometric indicators Web of Science (traditionally) Scopus (increasingly) Various field specific databases (less often, field studies) Google scholar (increasingly, but controversial) CRIS-databases (new options) 4

5. Database requirements for bibliometric indicators (simplified) RequirementsWoSScopusGoogle Scholar CRIStinCRIS (prior) Adapted for bibliometric studies + +-0- Complete indexing of all authors and author addresses ++-0, +- Citation counts+++-- Transparent and justified selection procedure of source material 0,+ -+- Coverage of natural sciences and medicine ++++0 Coverage of engineering00++0 Coverage of A&H and SS--++0 Coverage of books and monographs --++0 5

6. CRIS-data Norway Data on all types of scientific/scholarly publications in all fields of research are collected trough a documentation system (CRIStin). In the database the publication output is reported by the institutions as ordinary bibliographic references. A dynamic authority record of controlled scientific and scholarly publication channels ensures that references to non-scientific publications are not entered into the system. Publication data from professional bibliographic data sources (e.g. Web of Science) are imported to the documentation system in order to facilitate the registration of publications by the employees.

7. Database requirements for bibliometric indicators (simplified) RequirementsWoSScopusGoogle Scholar CRIStinCRIS (prior) Adapted for bibliometric studies + +-0- Complete indexing of all authors and author addresses ++-0, +- Citation counts+++-- Transparent and justified selection procedure of source material 0,+ -+- Coverage of natural sciences and medicine ++++0 Coverage of engineering00++0 Coverage of A&H and SS--++0 Coverage of books and monographs --++0 7

9. Database requirements for bibliometric indicators (simplified) RequirementsWoSScopusGoogle Scholar CRIStinCRIS (prior) Adapted for bibliometric studies + +-0- Complete indexing of all authors and author addresses ++-0, +- Citation counts+++-- Transparent and justified selection procedure of source material 0,+ -+- Coverage of natural sciences and medicine ++++0 Coverage of engineering00++0 Coverage of A&H and SS--++0 Coverage of books and monographs --++0 9

10. Publication types* in five major areas, Norwegian universities *) Weighted: 1 book = 5 articles (N=24,154)

11. Coverage of 50 000 journal articles by Norwegian authors* 11 *) Accredited scientific and scholarly journals, CRIStin-data 2005-2012, Source: Gunnar Sivertsen (preliminary analysis)

12. Proportion of WoS indexed publications, selected SSH-departments, University of Bergen

13. Common indicators in bibliometric studies of research IndicatorData Research outputPublications per unit Research profilePublications per field Scientific impactNumber of citations Research collaborationCo-authorship 13

14. Addresses are the basis for indicators of: -Publication output per country or research unit

15. Total publication output 2012 (WoS) 15

16. National publication profile biology (Norway)

19. Counting of publications – methodological issues The question of how to handle papers involving co-authorship in general and international co-authorship in particular has been a recurrent issue in the literature on the use of bibliometric indicators Whole counting and fractional counting have been considered complementary methods According to Moed (2005), the integer or whole count method can be interpreted to measure participation, while the fractional counting method measures the number of papers creditable to a country. 19

20. Counting of publications – methodological issues Introduction of additional weighting systems, e.g. of first-authorship Weighting of books Publication types to be included 20

21. Publication channels - two levels Level 2- is giving extra weight includes only the leading and most selective international journals and publishers - 20 % of the total Level 1 80% of publications -”normal” wheight In the Norwegian system journals and publishers are divided into two levels

22. Funding model Norway: Weigted publication points based on publication level and publication channel: Level 1 Level 2 –Articles in scientific/scholarly journals: 1 3 –Articles in scientific/scholarly books: 0.7 1 –Monographs: 5 8

23. Common indicators in bibliometric studies of research IndicatorData Research outputPublications per unit Research profilePublications per field Scientific impactNumber of citations Research collaborationCo-authorship 23

24. Journal titles are the basis for indicators of: - Allocation of publications to fields of research - Research profiles - Field normalization

25. Number of articles* by field, 2005-2009 *) Indexed in Web of Science 25

26. Relative Specialization Index for ‘traditional’ universities (blue line equals the world average) (2005-09) UiB UiO 26 NTNU UMB

27. Common indicators in bibliometric studies of research IndicatorData Research outputPublications per unit Research profilePublications per field Scientific impactNumber of citations Research collaborationCo-authorship 27

28. Citations are the basis for indicators of: - Impact (field normalized)

29. Citation rates by field (2005, 5 year-window) DisciplineCit/pubUncitedDisciplineCit/pubUncited Molecular Biology & Genetics22.44 %Geosciences8.812 % Immunology20.73 %Physics8.417 % Microbiology15.64 % Plant & Animal Science6.916 % Biology & Biochemistry15.17 %Agricultural Sciences6.916 % Space Science14.213 %Economics-Business5.620 % Clinical Medicine12.510 % Social Sciences, general5.022 % Chemistry10.713 %Engineering4.626 % Environment/Ecology10.39 %Mathematics3.329 % Psychiatry/Psychology10.211 %Computer Science2.948 %

30. Number of citations per year per article

31. Cumulative share of citations and publications, % (2005 - 5 years cit. win)

33. Citations counts – different sources ARTL_TITLEJRNLFULLVOLYEAR Citations - Web of ScienceCitations - Scopus Citations - Google scholar A macro study of self-citationSCIENTOMETRICS56200390102170 Characteristics of highly cited papers RESEARCH EVALUATION122003576295 Peer reviews and bibliometric indicators: a comparative study at a Norwegian university RESEARCH EVALUATION1320044954109 Does self-citation pay?SCIENTOMETRICS722007414866 Scientific productivity and group size: A bibliometric analysis of Norwegian microbiological researchSCIENTOMETRICS492000323857 The effect of highly cited papers on national citation indicatorsSCIENTOMETRICS592004303258 Citation rates and perceptions of scientific contributionJASIST572006293774 Validation of bibliometric indicators in the field of microbiology: A Norwegian case studySCIENTOMETRICS492000161318 Researchers' perceptions of citationsRESEARCH POLICY382009151721 When different persons have an identical author name. How frequent are homonyms?JASIST59200891214 The Structure and Development of Polar Research (1981- 2007): a Publication-Based Approach ARCTIC ANTARCTIC AND ALPINE RESEARCH412009448 Ranking national research systems by citation indicators. A comparative analysis using whole and fractionalised counting methods JOURNAL OF INFORMETRICS620124610 Are Female Researchers Less Cited? A Large-Scale Study of Norwegian Scientists JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE AND TECHNOLOGY622011239 Science policy and the driving forces behind the internationalisation of science: The case of Norway Science and Public Policy35200848 Total378432717

34. Google scholar vs Web of science

35. Relative citation index Relative citation index field = (Observed Citation Rate/World Citation Average of Subfield)X100 Relative citation index journal = (Observed Citation Rate/Average citation rate of journal)X100 Example (shown for one article only): –A regular journal article in Acta Chrystallographica E, 2005 has been cited 3 times. –The articles published in Acta Chrystallographica E were in contrast cited 1.74 times on average this year. –The Relative citation index – journal is: (3/1.74)*100 = 172. –The world-average citation rate for the subfield which this journal is assigned to is 3.7 for articles published this year. The Relative citation index – field is: (3/3.7)*100 = 81

36. Field normalized relative citation indexes, 2008-2011 36

37. Field normalized relative citation indexes of universities and university hospitals (2005-2008) across subject fields 37

38. H-index Articles ranked by decreasing citation numbers. A scientist with an index of h has published h papers each of which has been cited by at least h times –can be manually determined using citation databases or using automatic tools –attempts to measure both the scientific productivity and the apparent scientific impact of a scientist –Reduces the effect of highly cited papers –But not without shortcomings: No field normalisation. H [read: age] index.

39. Journal impact factor One of the most well-known bibliometric indicators Originally introduced by Eugene Garfield as a measure of the frequency with which the average article in a journal has been cited Average citation rate for 1 and 2 years old articles published in a particular journal: Number of citations received in year T to the articles published in a journal during the years T-1 og T-2 divided by the number of articles published in the journal during the years T-1 og T-2

40. Common indicators in bibliometric studies of research IndicatorData Research outputPublications per unit Research profilePublications per field Scientific impactNumber of citations Research collaborationCo-authorship 40

41. Addresses are the basis for indicators of: -Publication activity per university -Research collaboration (combinations of addresses in publications)

42. Map of scientific collaborations 2005 to 2009

43. Norwegian articles with and without international co- authorship, 1981-2009

44. The institutional national and international collaboration profile of the University of Bergen based on co-authorship

45. The international collaboration profile of the University of Bergen based on co-authorship

46. The institutional international collaboration profile of the University of Bergen based on co-authorship

47. Evaluation of research performance - an important application of bibliometrics Growing, often controversial, policy interest to use bibliometric techniques as output measures of research productivity and impact Bibliometrics evolved to a standard tool in science policy and research evaluation 47

48. Science policy applications Monitoring research systems Evaluations Funding

49. Applications in research evaluations Levels of analyses –Country –Institutions –Departments –Divisions/research groups –(Individuals – anonymous presentation)

50. Bibliometric indicators for funding purposes In 2004, Norway implemented a bibliometric model for performance based budgeting of research institutions –The funding of these institutions is now partially based on the measurement of their scientific and scholarly publishing –National bibliographic database (CRIStin) used for this purpose –Similar system for research institutes and hospitals

51. Questions – comments?

52. www.nifu.no