1. Influence of the Danish Performance Indicator on overall Danish Citation Impact 2000-12
Peter Ingwersen
Royal School of Library & Information Science, DK
clb798 at iva.ku.dk

2. Table of Contents Motivation & Research Questions
Royal School of Library and Information Science, Denmark
Table of Contents
Motivation & Research Questions
Introduction to the performance model in Denmark, BFI.
Methodlogy
Data collection methods
Web of Science
National statistics (performance system)
Findings
Overall developments of productivity, citations and impact across documents types
(Inter)national collaboration ratio and impact across document types
Discussion and Conclusions
IVA 2015

3. Royal School of Library and Information Science, Denmark
Motivation
Investigation of patterns of research productivity and citation impact across three document types prior to and after the
Introduction of the BFI performance assessment system 2008/09
Peer reviewed Research articles; Review articles; (3 or 1 point) Proceeding papers (not ‘meeting abstracts’)(.7 points; from 2013: like articles in 2 levels)
We expected:
Minor temprorary stand-still in productivity (uncertainty; resistance to BFI)
Decrease in collaboration due to fractionalization
IVA 2015

4. Royal School of Library and Information Science, Denmark
The current points (from 2014)
Publication type
Level 1
Level 2
Academic monographs (excl. teaching books) 5 8
Articles in journals; book & conf. series (incl. review articles; conf. papers; excl. abstracts; ed. comments; reviews) 1 3
Academic articles in anthologies (incl. book chapters; conf. papers not in series; excl. foreword; presentation manus; power points; encyclopaedic articles)
0.5 2
All journals, series and publishers must be on the authority lists (= peer reviewed); cooperative items fractionalized by institutions; min. fraction = 0.1, multiplied by 1.25
IVA 2015

5. Motivation and research questions
Royal School of Library and Information Science, Denmark
Motivation and research questions
Government hope: reinforce Danish research production & impact owing to better quality after 2008
Encourage publishing in 3-point Level 2 journals
Equalize humanities with science and health sciences
Research questions:
Did the introduction of the BFI performance indicator for research publications in 2008/09 alter the Danish productivity patterns, citation impact or (inter)national collaboration in the following years? (excluding the humanities)
IVA 2015

6. Research questions cont.
Royal School of Library and Information Science, Denmark
Research questions cont.
Did BFI alter the productivity and citation patterns for specific document types?
Note that the public funding of universities and research is fairly constant at .9 – 1.0 % of BNP during the period (neutral/controlled variable)
But increased admin and academic university staff since 2006!
(Proportion of admin staff increases)
Did the observed patterns follow the academic staff increase pattern?
IVA 2015

7. Methodology Data collection:
Royal School of Library and Information Science, Denmark
Methodology
Data collection:
April 21, 2013 in Web of Science: SSCI; SSCI; CPCI-S/SSH (HUM and books not taken into account)
For all the years the Danish share lies between .8 and 1.0 %
Annual impact analysis with 3-year citation windows ( )
Proceeding papers in SCI+SSCI are also tagged as ‘article’
Proceeding papers from CPCIs are ‘sometimes’ also tagged as ‘article’, the overlap is changing all the time and CPCIs’ coverage are quite variable with a definitive decline from 2008.
IVA 2015

8. Royal School of Library and Information Science, Denmark
Data collection cont.
All documents indexed both as ‘article’ and ‘proceeding paper’ were extracted as proceeding papers (excluded from the ‘article category) – they are sample checked as publised in thematic journal/serial issues.
WoS analytic tools were applied in order to isolate document types annually for a) international collaboration and b) national publications, as well as mean number of countries (in set a)) and Danish institutions (in set b));
Actual numbers of publication types were extracted from the Danish Res. Agency’s public data (on the web: )
IVA 2015

9. Indicators used Overall and for each document type 2000-2012:
Royal School of Library and Information Science, Denmark
Indicators used
Overall and for each document type :
Publication analysis (annual and periodic)
Citation analysis (annual and periodic)
Citation impact (annual and periodic)
Collaboration at Danish (institutional) level
Average number of institutions per document
International collaboration ratio (country level)
Average number of countries per document
Comparison of WoS data with national statistics of real Danish research output
Analysis of Web of Science coverage of proceeding papers
IVA 2015

10. Royal School of Library and Information Science, Denmark
Actual developments of doc. types (data: Danish Research Agency, 2014)
IVA 2015

11. Distributions of document types across academic fields
Royal School of Library and Information Science, Denmark
Distributions of document types across academic fields
IVA 2015

12. Academic staff increase, WoS vs. actual article increase
Royal School of Library and Information Science, Denmark
Academic staff increase, WoS vs. actual article increase
IVA 2015

13. Royal School of Library and Information Science, Denmark
International cooperation, countries and purely Danish authorship (WoS)
IVA 2015

14. Proc. / anthology papers
Royal School of Library and Information Science, Denmark
Int. Cooperation Ratio ( ), average no. of collaborating institutions in purely Danish documents
Research articles
Proc. / anthology papers
(WoS)
IVA 2015

15. Citation impact: purely Danish and international coop.
Royal School of Library and Information Science, Denmark
Citation impact: purely Danish and international coop.
(WoS)
IVA 2015

16. Royal School of Library and Information Science, Denmark
Findings: Annual Danish citation impact – 3-year window (WoS)
IVA 2015

17. Royal School of Library and Information Science, Denmark
Ratios of Danish Proceeding papers vs. Research articles in all WoS indexes and real (Danish Research Agency statistics) - WoS coverage
IVA 2015

18. Growth of WoS (CPCI-S/SSH) 2000-2012
Royal School of Library and Information Science, Denmark
Growth of WoS (CPCI-S/SSH)
IVA 2015

19. Distribution of Level 1 articles (Danish Agency data)
Royal School of Library and Information Science, Denmark
Distribution of Level 1 articles (Danish Agency data)
IVA 2015

20. Discussion & Conclusions
Royal School of Library and Information Science, Denmark
Discussion & Conclusions
Research Question 1 (Performance system goals from 2008/09):
Highly Positive effect on Research Article productivity (Agency statistics) :
In particular Level 2 journals:
Overall growth: 33.3 %
Science & Technology: 46 %
Social Sciences: 14 %
Health Sciences: 6.4 %
Level 1 journals: 27.8 %
Negative effect on Proceeding Papers: -14 %
IVA 2015

21. Discussion & Conclusions cont.
Royal School of Library and Information Science, Denmark
Discussion & Conclusions cont.
Research Question 1 (International cooperation):
Decline in international cooperation ratio for Proceeding Papers; and
Impact decline in International collaboration and in purely Danish papers;
Stability of international cooperation ratio for Research Articles :
Increase of impact for purely Danish articles (stability from 2008); stable no. of institutions;
Strong continuous increase of impact for international articles ; increased no. of coop. countries
IVA 2015

22. Discussion & Conclusions cont.
Royal School of Library and Information Science, Denmark
Discussion & Conclusions cont.
Research Question 2 (franctionalization issue since 2008/09):
Does not seem to influence behavior for Research Article production
(international cooperation ratios and no. of Danish institutions per doc. are stable – no. of cooperating countries increases!);
Fractionalization influences negatively productivity of Proceeding Papers:
Decline in international and national cooperation
IVA 2015

23. Discussion & Conclusions cont.
Royal School of Library and Information Science, Denmark
Discussion & Conclusions cont.
Research question 3: The productivity increase for articles is much greater than that for academic staff during the same period.
Research impact (question 1): Increase continues for research articles but decreases for proc. papers.
No indication of ’salami publishing tactics’
IVA 2015

24. CONCLUSIONS RESEARCH ARTICLE PUBLICATION BEHAVOIR:
Royal School of Library and Information Science, Denmark
CONCLUSIONS
RESEARCH ARTICLE PUBLICATION BEHAVOIR:
Influenced by the performance assessment system – also seen as a incentive in many institutions
Positive trends for Level 2 journal use
PROCEEDING PAPERS influenced by:
Performance assessment system negatively (less points);
Productivity and (inter)national cooperation (fractionalization)
IVA 2015

25. THANK YOU! Royal School of Library and Information Science, Denmark
IVA 2015