CERIF for research evaluation including REF Keith G Jeffery STFC / euroCRIS 20/04/20111Ready 4 Ref End of Project Workshop

Structure CERIF CERIF in Use Evaluation REF Results 20/04/2011Ready 4 Ref End of Project Workshop2

Project Person / CV Institution Event Equipment Books Journal/article Patent Research Group Publisher Some entities (objects) of interest

CERIF Model EU Recommendation to Member States

PROJECTORGUNITPERSON Result_Publication RESULT_PUBLICATION Concepts: (1) temporally-bound role linking relations (2) >1 linking relation : Result_Publication and other entities (3) PERSON role may be author, co-author, editor, reviewer…. (4) ORGUNIT role may be publisher, IPR or copyright owner, funder.. (5) PROJECT role may be the source of the idea

RESULT_PUBLICATION PROJECT ORGUNIT PERSON Result_Publication Can Express (assertions in pseudo-English): Person A (DT1 - DT2) (is author of) Publication X Orgunit O (DT1 - DT2) (is owner of IPR in) Publication X Person A (DT1 – DT2) (is employee of ) Orgunit O Person A (DT1 - DT2) (is project leader of) Project P Person A (DT1 - DT2) (is member of) Orgunit M Person A (DT1 - DT2) (is member of)Orgunit N Orgunit M (DT1 - DT2) (is part of) Orgunit O Orgunit N (DT1 - DT2) (is part of) Orgunit O

Result_Publication Instance Diagram Person A Publication X OrgUnit O OrgUnit M OrgUnit N Project P member employee Part of owns IPR author Project leader

CERIF-CRIS at One Organisation Publication repository Dataset Software repository Finance system Human Resources system Project Management system CERIF-CRIS Web pages Directory Services This is fine for one organisation but research is international, so…

CERIF Interoperation CERIF-CRIS CERIF provides interoperation of CRIS and associated systems with formal syntax and declared semantics so that it is reliable and scalable. Interconnect Backplane

CRIS + Repositories at 1 institution CRIS Research Context [projects, persons, organisational units funding, products, patents, publications facilities, equipment, events] OA Repository (hypermedia) Documents e-Research repository Datasets and Software OAI- PMH Various protocols End-User CERIF

….and multiple institutions CRIS OA repository e-Research repository CRIS OA repository e-Research repository CRIS OA repository e-Research repository End-User Institution AInstitution BInstitution C

Key Points Formal syntax and declared semantics Extensible yet backward-compatible Multilingual Interoperable Can generate semantic web EU Recommendation to member states 2000 Following independent technical and business case studies adopted for UK by RCs, HEFCE, UUK and others /04/2011Ready 4 Ref End of Project Workshop12

Structure CERIF CERIF in Use Evaluation REF Results 20/04/2011Ready 4 Ref End of Project Workshop13

CERIF in Use National systems: IS, NO, DK, NL, SK, SL, BE, RU (SE) In use in funding organisations and universities in many more countries: DE, FR, IT, FI, IE, GR, CZ In UK ~40 universities using it or working on it HEFCE, UUK, RCUK agreed (2010) CERIF is UK standard for research information Used for REF and for RC evaluation of research output Native CERIF-CRIS or CERIF-wrapped legacy CRIS

Structure CERIF CERIF in Use Evaluation REF Results 20/04/2011Ready 4 Ref End of Project Workshop15

Evaluation Research Environment – ‘standing’ including professional activities – ‘reputation’ of institution Quality and Quantity of Research – Securing research funding Outputs of Research – Scholarly works – Patents – Products Impact of Research – Wealth creation (including cost savings) – Improvement in the quality of life – Further research Funding Funding organisations Research organisations Facility managers Innovation managers Innovators Media / Public 20/04/2011Ready 4 Ref End of Project Workshop16

RC Evaluation RCS Fund ~£3B per year: peer reviewed proposals No final reports Assess outputs and outcomes Scholarly works in institutional repository – linked to grant reference (in CRIS) – Linked to organisational unit (in CRIS) – Linked to person (in CRIS) In time patents, products etc – At present only evidence is case studies 20/04/2011Ready 4 Ref End of Project Workshop17

HEFCE Evaluation HEFCE Funds ~£1.6B per year of which £1.3B is quality related by subject by institution From RAE to REF Still basic parameters – Quality of research – Research environment And adding more strongly – Impact of research 20/04/2011Ready 4 Ref End of Project Workshop18

Structure CERIF CERIF in Use Evaluation REF Results 20/04/2011Ready 4 Ref End of Project Workshop19

REF: Research Excellence Framework Not defined yet but 3 aspects: – quality of research, impact of research, research environment Likely to have similar parameters to RAE Expert Review plus indicators – Not totally automated based on measures Impact: historical not future Environment: vitality – Equalities and diversity /cl04_10a.pdf Use of HESA statistics Use of templates Expert panels 20/04/2011Ready 4 Ref End of Project Workshop20

Structure CERIF CERIF in Use Evaluation REF Results 20/04/2011Ready 4 Ref End of Project Workshop21

Results Initial assessment of CERIF for REF purposes at euroCRIS members’ meeting St Andrews November 2009 – Only 2 additional attributes needed – Principle of atomic attribute values and aggregation Ready 4 Ref project analysis and structure – Ensure required measures are in CERIF data model Only 2 additional attributes needed – Ensure use of CERIF XML for data interoperation demonstrated 20/04/2011Ready 4 Ref End of Project Workshop22

And looking forward MICE project new structure – Attempt to ensure measurable parameters for impact are in CERIF data model New group of entities with attributes Proposal from MICE to euroCRIS CERIF Task Group for discussion May Copenhagen 20/04/2011Ready 4 Ref End of Project Workshop23