Dr. Edriss Ali, AGU - Dubai 11/22/2018 Dr. Edriss Ali, AGU - Dubai Global University Ranking Methodologies and Their Impact on Higher Education Quality Dr. Edriss Ahmed Ali Associate Professor, College of Engineering & Computing, Al Ghurair University, Dubai, UAE edirss@agu.ac.ae

Dr. Edriss Ali, AGU - Dubai 11/22/2018 Dr. Edriss Ali, AGU - Dubai Outline Introduction An Outline of Popular Global Ranking Organizations: Global Ranking Systems Methodologies Why Global Rankings do not reflect the actual quality of Institution Implications of Ranking On Some Universities’ Status Institutional Quality Indicators Why Ranking Measures are More Informative than QA Measures: Proposed Methodology for Ranking Conclusions & Recommendations

Dr. Edriss Ali, AGU - Dubai 11/22/2018 Dr. Edriss Ali, AGU - Dubai Introduction A number of famous global ranking systems have recently appeared in the higher education sector, claiming that they compare the education quality of universities and higher education colleges. This was due to globalization of higher education and the implication they have on so social and economical development. There are three main levels of Ranking: National: at country level e. g. US national Regional: universities existing in a certain region, e. g. Arab World, Asian Region, Latin American, … etc. Global: at international level e.g. QS, THE, … etc. Discipline: e. g. Engineering, Business … etc.

Evolution of Ranking Systems National university rankings have been used for decade in the States (1905). Global rankings first appeared in 2003, when a team of researchers at Shanghai Jiatong University (China) produced the Academic Ranking of World Universities (ARWU) to benchmark Chinese universities with top universities in the World. In 2004 appeared Times Higher Education – QS World University Rankings (THE-QS). THE-QS split into Times Higher Education World University Ranking (THE) and QS World University Ranking (QS) in 2010. In 2009, a multidimensional mapping and ranking project funded by European Union called ‘U-Multirank appeared. Global rankings then grew over time and coincided with the advance of globalization and importance of higher education for social and economic development and marketization of higher education, and mobility of students and faculty.

An Outline of Popular Global Ranking Organizations: 11/22/2018 Dr. Edriss Ali, AGU - Dubai An Outline of Popular Global Ranking Organizations: we selected three famous international university rankers; namely: ARWU: (Academic Ranking of World Universities) by Shanghai Ranking Consultancies, THE : The Times High Education (World University Ranking), QS World University Ranking.

Data Sources of Global Ranking Systems Many global rankers, e. g. ARWU, THE , U-Multirank, use ‘Thomson Reuters Web of Science’ and ‘Thomson Reuters Incite’. Only QS draws its data from SCOPUS, a product of Elsevier, a Dutch-based data broker.

Ranking Methodologies 11/22/2018 Dr. Edriss Ali, AGU - Dubai Ranking Methodologies Indicator Weights: After selecting indicators, each is given a weight depending (on the opinion of the ranker) on its relative contribution to the HEI quality. Scope: In order to limit the number of HEIs, a criteria is set, such as the size of the institute, number of disciplines it covers, whether under-graduates only or both under and post graduates … etc.

Indicators and Weights for ARWU Criteria Indicator Code Weight Quality of Education Alumni of an institution winning Nobel Prizes and Fields Medals Alumni 10% Quality of Faculty Staff of an institution winning Nobel Prizes and Fields Medals Award 20% Highly cited researchers in 21 broad subject categories HiCi Research Output Papers published in Nature and Science* N&S Papers indexed in Science Citation Index-expanded and Social Science Citation Index PUB Per Capita Performance Per capita academic performance of an institution PCP Total 100%

Example: QS Ranking System 11/22/2018 Dr. Edriss Ali, AGU - Dubai Example: QS Ranking System Methodology of QS World University Rankings[16] Indicator Weighting Elaboration Academic peer review 40% Based on an internal global academic survey Faculty/Student ratio 20% A measurement of teaching commitment Citations per faculty A measurement of research impact Employer reputation 10% Based on a survey on graduate employers International student ratio 5% A measurement of the diversity of the student community International staff ratio A measurement of the diversity of the academic staff

Example THE Ranking System 11/22/2018 Dr. Edriss Ali, AGU - Dubai Example THE Ranking System

Comparison of indicators and their weights of three popular Rankers 11/22/2018 Dr. Edriss Ali, AGU - Dubai Comparison of indicators and their weights of three popular Rankers

Dr. Edriss Ali, AGU - Dubai 11/22/2018 Dr. Edriss Ali, AGU - Dubai Why Global Ranking Systems do not reflect the actual quality of Institution Indicators used are not inclusive Indicators used are biased toward a minor set of quality aspects Relative position in ranking table based on a single average weight Historical students background not considered Source of Indictors measurement data Not all Universities Considered

Implications of Ranking On Some Universities’ Status 11/22/2018 Dr. Edriss Ali, AGU - Dubai Implications of Ranking On Some Universities’ Status Due to many restrictions imposed by popular ranking systems, many universities in the developing world, especially some Arab region universities, do not find the chance of being considered for ranking, and if considered, are low ranked due to limited resources in fields like research, Investigation of the list of these universities and their origin shows that most of them are in the relatively rich GCC countries with few exception, where a reasonable amount of fund is allocated to research (King Abdel Aziz Center for Scientific Research in Saudi Arabia).

Dr. Edriss Ali, AGU - Dubai 11/22/2018 Dr. Edriss Ali, AGU - Dubai Arab World Universities in THE Global Raking (The Chronicle of Higher Education)  Country: University: Rank Saudi Arabia King Abdulaziz University 251-300 King Fahd University of Petroleum and Minerals 501-600 King Saud University UAE United Arab Emirates University Lebanon American University of Beirut Jordan University of Jordan 601-800 Jordan University of Science and Technology Qatar Qatar University Morocco University of Marrakech Cadi Ayyad American University of Sharjah Egypt Alexandria University Cairo University Suez Canal University Oman Sultan Qaboos University

Institutional Quality Indicators 11/22/2018 Dr. Edriss Ali, AGU - Dubai Institutional Quality Indicators A number of researchers investigated identification a possible set of quality indicators in higher education, among these the work of the Educational Policy Institute of Higher Education Quality Council [10], in which the areas for quality indicators are categorized into five categories as: Input Indicators (no. students enrolled, no. faculty, FT/PT) Research Indicators (budget, publications, citations .. Etc.) Teaching and Learning Indicators (cognitive development, satisfaction surveys both students, employers), Internal Service Indicators (completion, continuation rates, employment rates … etc.) Output Indicators (graduate students satisfaction, exit survey, spending on services, libraries, network access … etc.).

Why Ranking Measures are More Informative than QA Measures: 11/22/2018 Dr. Edriss Ali, AGU - Dubai Why Ranking Measures are More Informative than QA Measures: Parameter QA Systems Ranking Systems Purpose To guarantee compliance with (minimum) standards To identify “Excellence” has after commercial purpose Method of Implementation Public framework or Independent non-profit organization Private Companies Consequences of Outcome It is mandatory for operation of an Institute or Program No legal consequences Type of Input Data Data have to be collected through teaching and learning process Use research-related and other criteria for which data are available Nature of Output Report Hard to understand by a layman Results are easily readable and understandable Relation to quality of Education Geared towards measuring or indicating education quality Useful for measuring only certain aspects of quality of education Cost Relatively high Low and sometimes no cost apart from that required for collecting available data

Proposed Methodology for Ranking 11/22/2018 Dr. Edriss Ali, AGU - Dubai Proposed Methodology for Ranking In order to find appropriate quality indicators that lead to a combined aggregate values for a higher education institutes we first look at main functions of any HEI, which are traditionally grouped into three main areas; Teaching and Learning, Research and Other Scholarly activities, and Community Service (engagement). All educational and related activities of any HEI can be associated to one of these three main areas. Hence if quality indicators for each activity related to each of these three areas, an aggregate quality indicator in each can be estimated, and hence an aggregate indicator for the HEI can approximately be estimated.

Proposed Methodology for Ranking (The Conceptual Model) 11/22/2018 Dr. Edriss Ali, AGU - Dubai Proposed Methodology for Ranking (The Conceptual Model)

Computation of total grade for each of the three categories (EDU, RES, Com) 𝑬𝑫𝑼» 𝒊=𝟏 𝒏 𝒆 𝒆 𝒊 = 𝑵 𝒆 𝑹𝑬𝑺» 𝒊=𝟏 𝒏 𝒓 𝒓 𝒊 = 𝑵 𝒓 𝑪𝑶𝑴» 𝒊=𝟏 𝒏 𝒄 𝒓 𝒊 = 𝑵 𝒄 𝑵 𝒆 + 𝑵 𝒓 + 𝑵 𝒄 ≤100

Conclusions & Recommendations 11/22/2018 Dr. Edriss Ali, AGU - Dubai Conclusions & Recommendations The mythologies and type of indicators used by three leading global university ranker have been considered and their shortcomings and the implications they have on educational quality have been identified. The starting point for ranking should be based on the outcome of Quality Assurance metrics followed by accreditation agencies, with appropriate indicators and their weights being chosen carefully. A ranking methodology that takes into account all university activities has been proposed. The need for a regional university ranking system that avoids all flaws of the existing ranking systems, and gives a true measure of HEI quality, based on quality indicators covering all dimensions of higher education quality, has been shown. This will give a fair judgment of educational quality of institutions, and in-line with AROQA mission statement and goals.