Recent developments in rankings: implications for developing countries? Jamil Salmi The World Bank IREG-3 Shanghai, October 2007

The rankings business A ranking of league tables September 10, 2005

Rankers, Beware League tables under scrutiny Gordon’s Brown’s disastrous retreat How do super-jumbo changes air travel Meet Fred Thompson Has commercial property peaked? Liu Nian Cai, the man and the myth

outline of the presentation status of rankings status of rankings new directions in accountability new directions in accountability prospects for ranking prospects for ranking

who prepares the rankings? A = government agency (Ministry of Higher Education, Higher Education Commission, University Grants Council, etc.) A = government agency (Ministry of Higher Education, Higher Education Commission, University Grants Council, etc.) B = independent organization / professional association / university B = independent organization / professional association / university C = newspaper / magazine / media C = newspaper / magazine / media D = accreditation agency D = accreditation agency I = International ranking (IA, IB, IC and ID linking the international dimension to the type of institution conducting the ranking) I = International ranking (IA, IB, IC and ID linking the international dimension to the type of institution conducting the ranking)

ranking systems in 2006 Region National and International Ranking System Eastern Europe and Central Asia Poland (C), Slovakia (B), Russia (B), Ukraine (B) East Asia and Pacific Australia (B), China (B, IB), Hong Kong (C), Japan (C), New Zealand (A), Thailand (A) Latin America and the Caribbean Argentina (D) Middle East and North Africa North America Canada (C), United States (C) South Asia India (D), Pakistan (A) Sub-Saharan Africa Nigeria (A) Western Europe Germany (B/C), Italy (C), Netherlands (A), Spain (B), United Kingdom (A, B, IC)

ranking systems in 2007 Region National and International Ranking System Eastern Europe and Central Asia Kazakhstan (A, B), Poland (C), Slovakia (B), Romania (B/C), Russia (B), Ukraine (B/C) East Asia and Pacific Australia (B), China (B, C, IB), Hong Kong (C), Japan (B, C), Korea (A), Malaysia (A), New Zealand (A), Taiwan (B), Thailand (A) Latin America and the Caribbean Argentina (D), Brazil (A), Chile (C,D), Mexico (B), Peru (B) Middle East and North Africa Tunisia (A) North America Canada (B, C, B/C), United States (C, IC) South Asia India (C/D), Pakistan (A) Sub-Saharan Africa Nigeria (A) Western Europe Germany (B/C, C), Italy (C), Netherlands (A), Portugal (C), Spain (B, C, IC), Sweden (C), Switzerland (B/C), United Kingdom (A, B, IC)

trends more rankings more rankings –except Africa and Middle East who does the rankings? who does the rankings? –less from the press –more from independent think tanks or governments accepted in growing number of countries accepted in growing number of countries voluntary participation voluntary participation –Austrians, Swiss, Flemish

outline of the presentation status of rankings status of rankings new directions in accountability new directions in accountability

what does accountability mean? information achievement of results sanctions

trends in accountability multiple stakeholders multiple themes multiple instruments

multiple stakeholders government(s) employers society at large professors students

multiple themes access equity quality relevance efficiency sustainability nation-building / values

multiple instruments licensing evaluation / accreditation performance-based funding rankings assessment of learning outcomes

measurement of outcomes From inputs to competencies – –Bologna process – –ABET Spellings Commission – –”no graduates left behind” OECD / PISA for tertiary education World Bank benchmarking tool World Bank benchmarking tool

country experiences Australia Brazil Canada Colombia Jordan Korea Mexico UK USA

outline of the presentation status of rankings status of rankings new directions in accountability new directions in accountability prospects for ranking prospects for ranking

usefulness of rankings? for the public? for the public? for the institutions? for the institutions? for the Government? for the Government?

usefulness of rankings: for individuals choice based on information

usefulness of rankings: for institutions benchmarking and self- evaluation counter-rankings (Paris) voluntary accountability framework (USA)

from the viewpoint of institutions sensitive to factors that affect their rankings (benchmarking) sensitive to factors that affect their rankings (benchmarking) goal setting for strategic planning purposes goal setting for strategic planning purposes forming strategic partnerships forming strategic partnerships mergers mergers

usefulness of rankings: for governments rapid QA assessment in lieu of accreditation international benchmarking national benchmarking culture of competition

government use of rankings Pakistan case Pakistan case –promoting a culture of accurate and transparent information –promoting a culture of quality

new challenges multiplicity of institutions (and joint programs) multiplicity of institutions (and joint programs) multiplicity of missions multiplicity of missions blurring of boundaries between disciplines blurring of boundaries between disciplines emerging disciplines emerging disciplines new delivery modes new delivery modes

conclusion: divisive or helpful?

For sale: charming and peaceful residence away from neighbors, with a wonderful view of the sea, a grand period staircase, and lots and lots of light…”

Bordeaux the wine producers suing the wine rankers

divisive or helpful? rankings are here to stay useful for prospective students useful in the absence of an established evaluation and/or accreditation system useful for benchmarking, goal- setting and self-improvement purposes useful to conduct a healthy debate on issues and challenges useful to promote a culture of accountability

linking rankings and funding? Arizona State Arizona State RAE (UK, Hong Kong, NZ, Australia) RAE (UK, Hong Kong, NZ, Australia) Nigeria (based on ranking done by Accreditation Commission) Nigeria (based on ranking done by Accreditation Commission) performance premium for academics publishing in journals included in Shanghai ranking (Norway, Australia) performance premium for academics publishing in journals included in Shanghai ranking (Norway, Australia) donors and home ministries making scholarship decisions donors and home ministries making scholarship decisions corporate donors corporate donors strengthening the strong and weakening the weak strengthening the strong and weakening the weak

principles of appropriate ranking instruments compare similar institutions compare similar institutions focus on program rather than on entire institution focus on program rather than on entire institution rank by indicator rather than wholesale rank by indicator rather than wholesale outcomes/outputs/results rather than inputs outcomes/outputs/results rather than inputs better if used for self-improvement purposes better if used for self-improvement purposes make results publicly available make results publicly available