1. www.che.de What Can National Rankings Learn from the U-Multirank-Project ? Gero Federkeil, CHE, Germany IREG-Forum: National University Rankings on the Rise Bratislava, 10-11 Oct. 2011

2. 2 IREG Forum | Gero Federkeil | Bratislava 2011-10-10

3. 3 The project Commissioned by the European Commission 2-year project, 2009 – June 2011 Report now available: http://ec.europa.eu/education/higher-education/doc/multirank_en.pdf Ján Figel, the former European Commissioner for Education, Training, Culture and Youth: “- to allow stakeholders to make informed choices; - to help institutions to position themselves and improve their performance” Two phases: o Design of new instrument o Testing the feasibility of new instrument

4. IREG Forum | Gero Federkeil | Bratislava 2011-10-10 4 Specification of U-Multirank Five dimensions: o Teaching & learning o Research o Knowledge transfer o International orientation o Regional engagement Long list of indicators to be tested in pilot project development of data collection tools and processes (question- naires, definitions, FAQs, communication + feedback processes) methods for building ranking groups instead of league tables

5. IREG Forum | Gero Federkeil | Bratislava 2011-10-10 5 Testing U-Multirank Two levels: Institution (FIR) Fields (FBR) Global sample of higher education and research institutions: 159 (target: 150), 2/3 Europe, 109 completed institutional questionnaires Two fields: Business studies Engineering (electrical and mechanical)

6. Bais logic: Mapping Diversity IREG Forum | Gero Federkeil | Bratislava 2011-10-10 6 Diversity of higher education institutions in Europe & the world Identifying comparable institutions that can be compared in one ranking Description of horizontal diversity  Types/profiles Description of horizontal diversity  Types/profiles Assessment of vertical diversity  Performance Assessment of vertical diversity  Performance Complementary instruments of transparency +

7. Mapping and Ranking IREG Forum | Gero Federkeil | Bratislava 2011-10-10 7 Mapping: Selection of a comparable set of universities based on institutional profiles Teaching and learning Research involvement Knowledge exchange Regional engagement International orientation Student profile Example: Comprehensive, teaching oriented institution Mainly undergraduate education Low research orientation Low international orientation Regionalyl embedded (e.g. recruiting) Subset of comparable institutions to be compared in a ranking

8. Mapping and Ranking IREG Forum | Gero Federkeil | Bratislava 2011-10-10 8 Ranking: Multi-dimensional ranking for subset of institutions No composite indicator! No number 1 !

9. 9 IREG Forum | Gero Federkeil | Bratislava 2011-10-10

10. Mapping and Ranking IREG Forum | Gero Federkeil | Bratislava 2011-10-10 10 Most national HE systems are diversified HE systems: Different types/profiles of institutions exist  Need to identify comparable institutions for ranking  Mapping systems can increase the comparabiliy and improve the quality of rankings  U-Map defines indicators for mapping & is setting a standard for Europe

11. Multi-dimensional Approach IREG Forum | Gero Federkeil | Bratislava 2011-10-10 11 Multi-Multirank identified a set of indicators for 5 dimensions U-Multirank introduced 2 „new“ dimensions: knowledge transfer regional engagement Indicators have been discussed intensively with stakholders

12. Innovative indicators IREG Forum | Gero Federkeil | Bratislava 2011-10-10 12 Teaching and learning: For rankings which want to inform (prospective) students indicators based on students‘ assessment of their teaching and learning experience are highly useful and are feasible (in most settings) Knowledge transfer: Joint publications with industry Research funds from industry But problems with regard to data (e.g. on spin offs/licenes)

13. Innovative indicators IREG Forum | Gero Federkeil | Bratislava 2011-10-10 13 International Orientation Rating indicator on international orientation of programmes is more meaningful than linear ranking of number of int. students Regional Engagement: Important for many HEIs yet most problematic dimension in U-Multirank Bibliometric indicator: Regional co-publications Further development is necessary

14. User-driven Approach IREG Forum | Gero Federkeil | Bratislava 2011-10-10 14 Intensive stakeholder consultation helped to increase acceptance Multi-dimensional, personalised rankings allow individual users to produce ranking based on their own preferences and networks and aossciations of universities to start benchmarking / create their own ranking

15. Data collection IREG Forum | Gero Federkeil | Bratislava 2011-10-10 15 International rankings have to rely on self-reported data due to lack of international data bases (except bibliometric, patent data) Feedback loop with universities concerning self-reported data on institution, faculties & programmes helped to increase consistency & quality of data Parallel / conflicting national data collections (e.g. student surveys)  raises issue of coordination national – international rankings in general

16. 16 IREG Forum | Gero Federkeil | Bratislava 2011-10-10

17. Outlook: National rankings and U-Multirank IREG Forum | Gero Federkeil | Bratislava 2011-10-10 17 There will be a continuing demand for national rankings ! Definition of a core set of indicators for national rankings and U-Multirank? Network of national rankings, e.g. Germany – Austria – Switzerland - Netherlands – Spain …. that share data which can be used for U-Multirank

18. 18

19. www.che.de What Can National Rankings Learn from the U-Multirank-Project ? Gero Federkeil, CHE, Germany IREG-Forum: National University Rankings on the Rise Bratislava, 10-11 Oct. 2011