November 2005Anne Asserson Research Administration Department,UiB How can you assess your organisation through a CRIS? The Norwegian national CRIS, FRIDA, and evaluation by means of research output

November 2005Anne Asserson Research Administration Department,UiB Structure of Presentation Ministry Model for Assessing Research Output Research Output – Publications Requirements of a CRIS in this context System Solution Evaluation of Output Ensuring Quality Conclusion

November 2005Anne Asserson Research Administration Department,UiB Aim of Ministry for Assessment Model for assessment based on indicators to Stimulate research and Allocate resources The best researchers (by allocating money based on publications) The best ideas (allocating money to the competition-based financing of projects) Allocate money to special thematic areas

November 2005Anne Asserson Research Administration Department,UiB New efforts: to use result indicators for academic/scientific publishing as a basis to budget the research at Universities and State Colleges. The Publications from 2004 will be included in the budgets for 2006 Main issues: 1.Provide better data on Academic Publishing 2.Take in consideration different practices – thus avoiding unfortunate effect on the publication pattern. 3.Divide in Quality levels to avoid reduction in quality of the publications pattern and avoid volume of low publications 4.Develop indicators for activities within dissemination St.prp.nr.1, UFD, pg. 115: ”Financing of Research” 2003

November 2005Anne Asserson Research Administration Department,UiB Ministry of Education and Research wish to improve research results in the universities Measure through indicators that say something of quality of results Results will trigger resource allocation to the universities 14% of total budget allocated based on research output indicators

November 2005Anne Asserson Research Administration Department,UiB Result Indicators that have been chosen –Doctoral theses –EU funding –National Research Council Funding –Publications These are quality indicators; all subject to peer review

November 2005Anne Asserson Research Administration Department,UiB Institution receives an allocation from the ministry The model is NOT recommended for use internally The ministry stresses the need for creating internal models to reward / stimulate Necessary to think complementarily in order to support particular subject areas

November 2005Anne Asserson Research Administration Department,UiB Structure of Presentation Ministry Model for Assessing Research Output Research Output – Publications Requirements of a CRIS in this context System Solution Evaluation of Output Ensuring Quality Conclusion

November 2005Anne Asserson Research Administration Department,UiB

November 2005Anne Asserson Research Administration Department,UiB Definition of an academic Publication The Publication must: Present new insight/knowledge Be in a form that make it possible to validate or useful in new research Be in a language and have a distribution that can make the publication available for the researchers of whom it may be of interest Be in a publication channel ( journal, series, publisher, website) with routines for Peer review.

November 2005Anne Asserson Research Administration Department,UiB Publications –Peer reviewed articles –Academic monographs –Chapters / parts of anthologies –(Proceedings) Only a few classes of publications used in assessment

November 2005Anne Asserson Research Administration Department,UiB Structure of Presentation Ministry Model for Assessing Research Output Research Output – Publications Requirements of a CRIS in this context System Solution Evaluation of Output Ensuring Quality Conclusion

November 2005Anne Asserson Research Administration Department,UiB Functionality Required Must be able to express : –Person belonging to one or more organisational unit –The output produced at one or more organisational units –from one year or a temporal interval –From one or several projects And these in any combination

November 2005Anne Asserson Research Administration Department,UiB Functionality Required 2 –Person must be linked to publication, organisational unit and project –Reference format Vancouver, APA etc –Person must be linkable to two or more organisational units

November 2005Anne Asserson Research Administration Department,UiB Structure of Presentation Ministry Model for Assessing Research Output Research Output – Publications Requirements of a CRIS in this context System Solution Evaluation of Output Ensuring Quality Conclusion

November 2005Anne Asserson Research Administration Department,UiB Structure Required relational model as basis with dynamic role-based temporal links between base entities Extended with elements of publication –Journal name, ISSN, shortname, acronym, pagefrom, pageto, volume, issue

November 2005Anne Asserson Research Administration Department,UiB PROJECTORGUNITPERSON SkillsCV General Facility Particular Equipment Contact Results Publication Results Patent Results Product Service Funding Programme Event Classification The Whole Thing Prize/Award

November 2005Anne Asserson Research Administration Department,UiB PROJECT ORGUNIT PERSON Result_Publication Can Express: (where DT-date/time) Person A (DT1 - DT2) (is author of) Publication X Orgunit O (DT1 - DT2) (is owner of IPR in) Publication X Person A (DT1 - DT2) (is employee of ) Orgunit O Person A (DT1 - DT2) (is project leader of) Project P Person A (DT1-DT2) (is member of) Orgunit M Person A (DT1-DT2) (is member of) Orgunit N Orgunit M (DT1-DT2) (is part of) Orgunit O Orgunit N (DT1-DT2) (is part of) Orgunit O Secondary Base Entities: example: RESULT_PUBLICATION

November 2005Anne Asserson Research Administration Department,UiB National system for the four universities Funded and owned by the universities Central database Based on CERIF (with extensions) 300,000 records in 2006 In production use for 1 year

November 2005Anne Asserson Research Administration Department,UiB Structure of Presentation Ministry Model for Assessing Research Output Research Output – Publications Requirements of a CRIS in this context System Solution Evaluation of Output Ensuring Quality Conclusion

November 2005Anne Asserson Research Administration Department,UiB A publications is divided in author and institutional parts within co-authoring Publication Author 1 Institution 1 Author 2 Institution 1 Institution 2 Author 3 Institution 3

November 2005Anne Asserson Research Administration Department,UiB Channel Form Eexample : scientific article in Nature or scientific monografi at Aschehoug forlag We find Academic Publications in the intersection between Publications Channel and Publication Form.

November 2005Anne Asserson Research Administration Department,UiB Level 1 and Level 2 At level 2 there are specified certain publication channels within all subjects. Level 2 are expressing the desirable direction for the publication pattern. Level 1 is the ”normal” and includes all publishing according to the definition.

November 2005Anne Asserson Research Administration Department,UiB Level 2: ca. 1/5 of the publications Level 1: ca. 4/5 of the publications

November 2005Anne Asserson Research Administration Department,UiB Guidelines for nomination at level 2 In general the publication channel must be Considered as leading within broader scientific community Publish the most important publications for researchers from different countries

November 2005Anne Asserson Research Administration Department,UiB Structure of Presentation Ministry Model for Assessing Research Output Research Output – Publications Requirements of a CRIS in this context System Solution Evaluation of Output Ensuring Quality Conclusion

November 2005Anne Asserson Research Administration Department,UiB Quality Import of ISI records –Problem: person and organisational unit not connected Routines and responsibility for quality assurance –Researchers –Organisational unit Researcher as end-user validates records

November 2005Anne Asserson Research Administration Department,UiB Bibliographical sources ISI-indexes will cover 62 % of all journal articles. Norart can cover further 32% of the articles by adjusting the application and by Nordic cooperation. Bibsys could possible cover 70-80% of book publishing

November 2005Anne Asserson Research Administration Department,UiB

November 2005Anne Asserson Research Administration Department,UiB Structure of Presentation Ministry Model for Assessing Research Output Research Output – Publications Requirements of a CRIS in this context System Solution Evaluation of Output Ensuring Quality Conclusion

November 2005Anne Asserson Research Administration Department,UiB Quality –Using selected publication channels, not only publication volume, gives quality Requirements of System –Cannot be done by a library system –Requires a flexible structured CRIS