UNICA Core Group on Evaluation in Research and Higher Education 29 October 2010 Vienna 1

I. Context 2 Data “mysteriously” aggregated Multidimensionality of research excellence neglected unreliable data etc. Current rankings not very useful for benchmarking or to develop research in one university

II. Objectives 3 to provide to the UNICA network a toolbox of research evaluation methodologies adapted to different “cases” identified as interesting by the UNICA members to pool resources in learning from each other about best practices in the context of benchmarking activities to develop a methodology accepted by Unica members, and based on expertise sharing, complementarities and harmonization of databases

III. Working Group 4 10 universities 4 meetings Carte d’europe et universités Leurs classements ectuels

IV. Methodology (1) 5 1. To list the possible objects to evaluate Projects Researchers Teams, faculties, centres, … … 2. To list the possible goals of an evaluation Typology Ranking Swot analysis … 3. To list the possible users of an evaluation Rectors Researchers Governments, funding agencies Media, public, … One case is defined by a combination of answers to each of the 3 questions

IV. Methodology (2) Focus on Case 2 Case study number ObjectGoalsUsers 1FacultiesStrong and weak pointsRectors 2Group/department (smaller groups than “faculties”) Strong and weak pointsRectors 3Teams, groups, faculties Strong and weak pointsPublic 6

III. Methodology (3) 7 Agreement to limit the number of indicators per dimension Dimensions : what do we want to take into account Indicators : how do we want to take into account Two first fields: Economic/business, Physics 5 Dimensions and 20 indicators D1: scientific output (7 indicators) D2: Attractiveness/internationalization (5 indicators) D3: Research training (3 indicators) D4: Transfer of technology (5 indicators) D5: Links to education (indicators still to be selected) + Basic Data on university (Global budget, staff, etc)

8 DIMENSION 1: SCIENTIFIC OUTPUT 1. Publications in Scopus, WoF 2. Peer review chapters of books 3. Peer review books 4. # Citations in databases 5. # Patents 6. # ERC grants 7. # Highly scientists III. Methodology (4)

IV. What has been done ? 9 Agreement on dimensions and indicators Data collection from several members of the group Analysis of difficulties

IV. Challenges and next steps 10 To get data from all members Analysis and graphical representation Need for common definitions of indicators Try to use existing data To have a meeting with Unica members involved in Multirank.