1. CRIStin, reporting and rewarding research
Eli Heldaas Seland and Monica Roos, 12 May 2016

2. Outline: The Norwegian system for weighted funding
Publication points as a principle for dividing funding between institutions
NSD – register of publication channels
Levels 1 and 2
Evaluation and recent changes to the system
Use and analysis of data
Cristin
Cristin and the library

3. The Norwegian Publication Indicator
System from 2004, for:
documenting Norwegian academic publishing
measuring publication activity
allocating research funding according to performance
Distributes approx. 2 % of total funds for university and university college sector
Evaluation in 2014 resulted in some changes

4. The original purpose of the indicator
…to create a performance-based funding model for research and to encourage more research activity at universities and university colleges. The documentation system must also be flexible so that it can be used for other reporting needs and in individual and institutional publications lists, applications, project reports and evaluations, annual reports and information for the public at large p.3.

5. Register of publication channels
Norwegian Register for Scientific Journals, Series and Publishers: authorized register of approved publication channels, divided into levels 1 and 2. A publication in a publication channel on this list, which fills specific criteria, is eligible to a certain amount of publication points – which in turn generate funding for the author’s institution (and, in many cases prestige for the authors)

6. What defines an academic publication?
An academic publication is defined according to four criteria; all four of these must be satisfied. The publication must:
present new insight;
be presented in a form that allows the research findings to be verified and/or used in new research activity;
be written in a language and have a distribution that make the publication accessible to most interested researchers;
appear in a publication channel (journal, series, book publisher, website) that has routines for external peer review.
p.12.

7. Levels 1 and 2
Level 1
scientific or academic publication channels (national and international)
Level 2
The most important / highly regarded publication channels, maximum 20% of a subject area’s total scientific publications
National academic councils/committees maintain the two levels. Anyone can nominate publication channels.

8. Weighting of publication types for the two levels of publication channels
Monograph 5 8
Article in periodical or series 1 3
Article in anthology
0.7

9. Fractionalization Originally:
A publication with 1 author is worth 1 point.
A publication with n authors is worth 1/n point for each author, but the value must be at least 1/10 point. (Fractions are not used when there are more than 10 authors.)
An institution is granted the sum of the point values from those authors who credit the institution.

10. 2014 evaluation: Has the Publication indicator stimulated more research and research of higher quality?
Publication volume/activity has increased, particularly with researchers and institutions that published little before
Other parameters remain stable
citation impact
national and international cooperation
little sign of salami publishing
little change in publication types
moderate shift from Norwegian to English language

11. Problems identified:
The increase in output has not been accompanied by an increase in impact
Lack of neutrality across disciplines: unintentional redistribution of funds between main areas
Legitimacy in nomination processes
Implementing incentives in institutions – exaggarated use at individual level

12. Solutions I
Stimulate cooperation, nationally and internationally, by introducing the factor 1.3 for any publication with an international co-author

13. Solutions II:
New way of calculating points that helps towards neutralizing across disciplines: √1/n

14. Solutions III New terminology
Points «earned» are more difficult to connect to individual authors
Discussions about local use is encouraged

15. What do we do?
Create reports for strategic use for the institution, and for faculties, on request
Prepare data for evaluations by the research council and other national committees and boards that ask for them
Train and advice researchers in and about the system

16. CRIStin is a national research information system for:
* Higher education sector
* Research Institutions
* Health sector
CRIStin is also an organisation responsible for lisences, the CRIS-system and they are the national Open Access agency.

17. One system composed by several parts:
One part to registrate activities (publications, conferences, music performances and so on)
One part for projects
One admin part for admin users
Several other parts which are not much used (research leave)

18. Serves two purposes for researchers and research:
As a «CV» of activities (researchers, but also admin staff)
Publications, seminars, conferences, music/theatre performances, workshops…)

19. 2. It is the tool used to do the actual «counting» of research publications:
When level publications (level 1 and 2) are registrated, they end up in a «list» sent once a year to the government.
- The library is responsible for administration, and the publication board for the professional evaluations.

20. Scientific material (articles, books and chapters) are the «countable» publications in the Norwegian weighted funding system.
They must be peer reviewed
They must be author affiliated
They must be written for, and distributed to peers.

21. Cristin and the Library I
The scientific material is checked and «approved» or «not approved» by the library
Based on the formal «rules» provided
Based on other institutions decisions
* If a publication has several Norwegian authors, all «home» institutions have to agree on an approval for the publication to actually be reported.
If a publication needs professional evaluation the publication board is doing this.

22. Cristin and the Library II
The library is the administrator of the system (Cristin manager)
We are training staff in registration and other uses of the system
We are checking the list of scientific publications and sending the report
We are in close cooperation with the research admin and the researchers
We are part of a national network, influencing how the cristin system should evolve

23. Cristin and the Library III
We are working with the communication department which is using cristin to harvest information about scientific publications and projects on to the HiB internal and external website
We are responsible for any national changes in the reporting instructions, making sure they are implemented to our local (HiB) routines
After the report is sent, the library starts working on the metrics, analysing this years numbers

24. Benefits for the library
We do metrics
We get a «say so» in digital work at HiB as Cristin is a master system on research in Norway
The leaders of the institutions must come to the library to ask for results
Researchers, teachers and others who are using the system must come to us
We know everything there is to know about scientific publication in Norway (and Europe?)
We gain knowledge on other related areas
We are involved in strategical research planning

25. Cristin 2.0
The CRIStin system is old, built on old technology and serving old needs in the sector.
A Cristin 2.0 is on its way. They are moving the system piece by piece over to new technology.
- They are aiming at a finish in 2018 – having a more flexible system

26. Links:
A Bibliometric Model for Performance-based Budgeting of Research Institutions Cristin