CESAER Task Force Benchmarking: SHARING / MONITORING / INFLUENCING 1 CESAER Task Force Benchmarking: SHARING / MONITORING / INFLUENCING Work, Results & Looking Ahead Special Focus: NTNU Prof. Dr.Techn. Mads Nygård Dean of Eng. Education, NTNU Chair, CESAER TF BENCH

The Happy Family – First Meeting in Copenhagen

CESAER TF Benchmarking: Background 3 Mandate: 2016-2018 (UMR => + THE, QS, ARWU & LEIDEN) Follow-up CESAER TF UMR Work on UMR New RMR Share Experiences and Learn from Each Other Monitor Tech Universities’ Ranking Performance Influence Rankers’ Measuring Methodologies Improve CESAER Members’ Strategies & Work 18 countries, 20 institutions, 28 members

CESAER TF Benchmarking: Activities 4

University Ranking Systems as Information Brokers

@UMultirank

U-Multirank: Several Dimensions + Ready Made Rankings U-Multirank: Several Dimensions + Ready Made Rankings ! Leiden: Several Variants - Select the Appropriate Ones ! THE Eng.: Slightly Adjusted Percentages U-Multirank Science & Tech.: Carefully Selected Indicators

The General Rankings: Not Adapted to Eng. / Science & Tech. (24.) (CES: 44) (CES: 30) (CES: 44)

THE Eng.: CESAER Performance 75% of CESAER members feature in the top 500 (41 vs. 44 / 51) 9

THE Eng.: 13 metrics – ALL Institutions vs. CESAER Institutions Left: ALL Right: CESAER 10

Strongest performing CESAER member in each country (top 10) Top 10 countries.

SWOT ANALYSIS: Top ranked CESAER Member in Each country Research Citation Score vs. Research Reputation Score 12

UMR READY MADE RANKING Science & Tech.: Indicators & Fields 13 Indicator   Level Institutional rankings Subject rankings TEACHING & LEARNING (4 / 9) Bachelor graduation rate X Master graduation rate Academic staff with doctorates Student-staff ratio RESEARCH (5 / 11) Art related output Citation rate Top cited publications Doctorate productivity Research publications (size normalized) KNOWLEDGE TRANSFER (4 / 9) Co-publications with industrial partners Publications cited in patents Spin-offs Patents awarded (size-normalized) INTERNATIONAL ORIENTATION (4 / 7) International academic staff International doctorate degrees International joint publications Student mobility REGIONAL ENGAGEMENT (0 / 6) No of Indicators 14 9 The Institutions & The 9 Subject Fields No of Entries All 231 Mathematics (NTNU: No 5) 67 Chemistry 91 Biology 48 Computer Science 104 Mechanical Engineering 102 Electrical Engineering 113 Chemical Engineering 77 Industrial Engineering 16 Civil Engineering 80 Which: CESAER Member / Tech NAME + > 40% PERC. Grad. & > 50% Valid INDIC.

(1.) 1. (2.) 2. (7.) 3. (12.) 4. (14.) 5. (15.) 6. (16.) 7. (17.) (20.)

UMR READY MADE RANKING Sci. & Tech. – 231 Institutions - 2017 15 CESAER Top 20 (among 231) DTU, DK (All: 2) Louvain, BE (All: 7) KTH, SE (All: 12) Delft, NL (All: 14) ETHZ, CH (All: 15) EPFL, CH (All: 16) Grenoble, FR (All: 17) Chalmers, SE (All: 21) Dublin, IE (All: 22) Eindhoven, NL (All: 25) Lund, SE (All: 26) München, DE (All: 32) Twente, NL (All: 33) Karlsruhe, DE (All: 35) Ghent, BE (All: 41) Aalborg, DK (All: 45) Technion, IL (All: 48) Aalto, FI (All: 56) Aachen, DE (All: 57) NTNU, NO (All: 66) CESAER Grouping (of 46/51) All 1-10: 2 (+2) All 1-20: 7 (+5) All 1-30: 11 (+4) All 1-40: 14 (+3) All 1-50: 17 (+3) All 1-60: 19 (+2) All 1-70: 20 (+1) All 1-80: 20 (+0) All 1-90: 23 (+3) All 1-100: 24 (+1) All 1-120: 28 (+4) All 1-140: 33 (+5) All 1-160: 36 (+3) All 1-180: 39 (+3) All 1-200: 42 (+3) All 1-220: 45 (+3) All 1-231: 46 (+1)

Averaging Own Indicators Only

Averaging Own Indicators Only - Readable (23.)

Averaging All 14 Indicators “Included”

Averaging All 14 Indicators “Included”- Readable (14.)

UMR READY MADE RANKING Science & Tech.: Different Sorting Crit. 20 With Relative No Method: DTU, DK (All: 2) Louvain, BE (All: 7) KTH, SE (All: 12) Delft, NL (All: 14) ETHZ, CH (All: 15) EPFL, CH (All: 16) Grenoble, FR (All: 17) Chalmers, SE (All: 21) Dublin, IE (All: 22) Eindhoven, NL (All: 25) With Absolute No Method: DTU, DK (Res: 11+2+0+0+0 = 13) Delft, NL (Res: 9+4+0+0+0 = 13) ETHZ, CH (Res: 9+3+1+0+0 = 13) EPFL, CH (Res: 9+3+0+1+0 = 13) Grenoble, FR (Res: 9+3+0+1+0 = 13) Eindhoven, NL (Res: 8+4+1+1+0 = 14) Chalmers, SE (Res: 8+1+1+2+0 = 12) Louvain, BE (Res: 5+2+0+0+0 = 7) KTH, SE (Res: 5+1+0+0+1 = 7) Dublin, IE (Res: 4+3+0+0+0 = 7) Averaging Own Indicators Only: DTU, DK (#: 13) Louvain, BE (#: 7) Delft, NL (#: : 13) ETHZ, CH (#: 13) Dublin, IE (#: 7) Grenoble, FR (#: 13) EPFL, CH (#: 13) München, DE (#: 13) Twente, NL (#: 13) Technion, IL (#: 7) Averaging All 14 Indicators “Included”: DTU, DK (#: 13 => 14) Delft, NL (#: 13 => 14) ETHZ, CH (#: 13 => 14) Grenoble, FR (#: 13 => 14) EPFL, CH (#: 13 => 14) München, DE (#: 13 => 14) Twente, NL (#: 13 => 14) Eindhoven, NL (#: 14 => 14) Ghent, IE (#: 13 => 14) Aalto, FI (#: 13 => 14)

UMR RMR – CESAER Performance Example: Research – Top 10 21

EU H2020 – CESAER Performance Example: ERC Grants – Top 15 22

CESAER External Profiling CESAER Internal Activity 23 Collective Size: Total No of Students: 1.036.806 Total No of International Students: 157.195 Total No of Academic Faculty Staff: 80.465 Individual Embedding: (By General Secretariat)

CESAER TF Benchmarking Past, Present & Future 24 Lessons Learned: Sharing & Learning Work Well – Also Here ! CESAER Members Perform Better – Especially on Eng. / Sci. & Tech. ! The Rankers may be Influenced – In Win-Win Situations ! Results Contain Structural Effects – As Well as Quality Effects ! Work Ahead: Continuous Cooperation – With Several Ranking Providers ! Increased Cooperation – With Other CESAER Task Forces ! From Ranking to Benchmarking – Data Analysis & Time Series ! Both External and Internal – Looking Better & Doing Better ! Inherent Challenges: Evolving & Expanding Landscape – Keeping Up-to-date ! What may be Measured – Vs. What should be Measured ! Subject & Data Definitions – Interpretation & Consistency Issues ! Subject & Data Results – Transparency & Accessibility !

University Mission Development: Current / Future Representat University Mission Development: Current / Future Representat. In Rankings

Questions? Comments / Reflections … 26 Questions? Comments / Reflections …