Influence of the Danish Performance Indicator on overall Danish Citation Impact 2000-12 Peter Ingwersen Royal School of Library & Information Science, DK clb798 at iva.ku.dk

Table of Contents Motivation & Research Questions Royal School of Library and Information Science, Denmark Table of Contents Motivation & Research Questions Introduction to the performance model in Denmark, BFI. Methodlogy Data collection methods Web of Science National statistics (performance system) Findings Overall developments of productivity, citations and impact across documents types (Inter)national collaboration ratio and impact across document types Discussion and Conclusions IVA 2015

Royal School of Library and Information Science, Denmark Motivation Investigation of patterns of research productivity and citation impact across three document types prior to and after the Introduction of the BFI performance assessment system 2008/09 Peer reviewed Research articles; Review articles; (3 or 1 point) Proceeding papers (not ‘meeting abstracts’)(.7 points; from 2013: like articles in 2 levels) We expected: Minor temprorary stand-still in productivity (uncertainty; resistance to BFI) Decrease in collaboration due to fractionalization IVA 2015

Royal School of Library and Information Science, Denmark The current points (from 2014) http://ufm.dk/forskning-og-innovation/statistik-og-analyser/den-bibliometriske-forskningsindikator/vaegtning Publication type Level 1 Level 2 Academic monographs (excl. teaching books) 5 8 Articles in journals; book & conf. series (incl. review articles; conf. papers; excl. abstracts; ed. comments; reviews) 1 3 Academic articles in anthologies (incl. book chapters; conf. papers not in series; excl. foreword; presentation manus; power points; encyclopaedic articles) 0.5 2 All journals, series and publishers must be on the authority lists (= peer reviewed); cooperative items fractionalized by institutions; min. fraction = 0.1, multiplied by 1.25 IVA 2015

Motivation and research questions Royal School of Library and Information Science, Denmark Motivation and research questions Government hope: reinforce Danish research production & impact owing to better quality after 2008 Encourage publishing in 3-point Level 2 journals Equalize humanities with science and health sciences Research questions: Did the introduction of the BFI performance indicator for research publications in 2008/09 alter the Danish productivity patterns, citation impact or (inter)national collaboration in the following years? (excluding the humanities) IVA 2015

Research questions cont. Royal School of Library and Information Science, Denmark Research questions cont. Did BFI alter the productivity and citation patterns for specific document types? Note that the public funding of universities and research is fairly constant at .9 – 1.0 % of BNP during the period (neutral/controlled variable) But increased admin and academic university staff since 2006! (Proportion of admin staff increases) Did the observed patterns follow the academic staff increase pattern? IVA 2015

Methodology Data collection: Royal School of Library and Information Science, Denmark Methodology Data collection: April 21, 2013 in Web of Science: SSCI; SSCI; CPCI-S/SSH (HUM and books not taken into account) For all the years the Danish share lies between .8 and 1.0 % Annual impact analysis with 3-year citation windows (2000-10) Proceeding papers in SCI+SSCI are also tagged as ‘article’ Proceeding papers from CPCIs are ‘sometimes’ also tagged as ‘article’, the overlap is changing all the time and CPCIs’ coverage are quite variable with a definitive decline from 2008. IVA 2015

Royal School of Library and Information Science, Denmark Data collection cont. All documents indexed both as ‘article’ and ‘proceeding paper’ were extracted as proceeding papers (excluded from the ‘article category) – they are sample checked as publised in thematic journal/serial issues. WoS analytic tools were applied in order to isolate document types annually for a) international collaboration and b) national publications, as well as mean number of countries (in set a)) and Danish institutions (in set b)); Actual numbers of publication types 2008-12 were extracted from the Danish Res. Agency’s public data (on the web: http://ufm.dk/ ) IVA 2015

Indicators used Overall and for each document type 2000-2012: Royal School of Library and Information Science, Denmark Indicators used Overall and for each document type 2000-2012: Publication analysis (annual and periodic) Citation analysis (annual and periodic) Citation impact (annual and periodic) Collaboration at Danish (institutional) level Average number of institutions per document International collaboration ratio (country level) Average number of countries per document Comparison of WoS data with national statistics of real Danish research output 2008-2011 Analysis of Web of Science coverage of proceeding papers IVA 2015

Royal School of Library and Information Science, Denmark Actual developments of doc. types 2008-2012 (data: Danish Research Agency, 2014) IVA 2015

Distributions of document types across academic fields Royal School of Library and Information Science, Denmark Distributions of document types across academic fields IVA 2015

Academic staff increase, WoS vs. actual article increase Royal School of Library and Information Science, Denmark Academic staff increase, WoS vs. actual article increase IVA 2015

Royal School of Library and Information Science, Denmark International cooperation, countries and purely Danish authorship (WoS) IVA 2015

Proc. / anthology papers Royal School of Library and Information Science, Denmark Int. Cooperation Ratio (0.0-1.0), average no. of collaborating institutions in purely Danish documents Research articles Proc. / anthology papers (WoS) IVA 2015

Citation impact: purely Danish and international coop. Royal School of Library and Information Science, Denmark Citation impact: purely Danish and international coop. (WoS) IVA 2015

Royal School of Library and Information Science, Denmark Findings: Annual Danish citation impact 2000-2010 – 3-year window (WoS) IVA 2015

Royal School of Library and Information Science, Denmark Ratios of Danish Proceeding papers vs. Research articles in all WoS indexes and real (Danish Research Agency statistics) - WoS coverage 2008-2011 IVA 2015

Growth of WoS (CPCI-S/SSH) 2000-2012 Royal School of Library and Information Science, Denmark Growth of WoS (CPCI-S/SSH) 2000-2012 IVA 2015

Distribution of Level 1 articles (Danish Agency data) Royal School of Library and Information Science, Denmark Distribution of Level 1 articles (Danish Agency data) IVA 2015

Discussion & Conclusions Royal School of Library and Information Science, Denmark Discussion & Conclusions Research Question 1 (Performance system goals from 2008/09): Highly Positive effect on Research Article productivity (Agency statistics) 2010-12: In particular Level 2 journals: Overall growth: 33.3 % 2009-12 Science & Technology: 46 % Social Sciences: 14 % Health Sciences: 6.4 % Level 1 journals: 27.8 % 2009-12 Negative effect on Proceeding Papers: -14 % IVA 2015

Discussion & Conclusions cont. Royal School of Library and Information Science, Denmark Discussion & Conclusions cont. Research Question 1 (International cooperation): Decline in international cooperation ratio 2009-12 for Proceeding Papers; and Impact decline in International collaboration and in purely Danish papers; Stability of international cooperation ratio for Research Articles 2000-12: Increase of impact for purely Danish articles 2000-2010 (stability from 2008); stable no. of institutions; Strong continuous increase of impact for international articles 2000-2010; increased no. of coop. countries IVA 2015

Discussion & Conclusions cont. Royal School of Library and Information Science, Denmark Discussion & Conclusions cont. Research Question 2 (franctionalization issue since 2008/09): Does not seem to influence behavior for Research Article production (international cooperation ratios and no. of Danish institutions per doc. are stable – no. of cooperating countries increases!); Fractionalization influences negatively productivity of Proceeding Papers: Decline in international and national cooperation IVA 2015

Discussion & Conclusions cont. Royal School of Library and Information Science, Denmark Discussion & Conclusions cont. Research question 3: The productivity increase for articles is much greater than that for academic staff during the same period. Research impact (question 1): Increase continues for research articles but decreases for proc. papers. No indication of ’salami publishing tactics’ IVA 2015

CONCLUSIONS RESEARCH ARTICLE PUBLICATION BEHAVOIR: Royal School of Library and Information Science, Denmark CONCLUSIONS RESEARCH ARTICLE PUBLICATION BEHAVOIR: Influenced by the performance assessment system – also seen as a incentive in many institutions Positive trends for Level 2 journal use PROCEEDING PAPERS influenced by: Performance assessment system negatively (less points); Productivity and (inter)national cooperation (fractionalization) IVA 2015

THANK YOU! Royal School of Library and Information Science, Denmark IVA 2015