1. Positional impact of reputational surveys in global university rankings
Philippe Vidal and Ghislaine Filliatreau
Observatoire des Sciences et Techniques, Paris
IREG Forum University Rankings, Warsaw May 2013

2. IREG Forum University Rankings, Warsaw 16 -17 May 2013
Reputation is an idle and most false imposition, oft got without
merit, and lost without deserving
Shakespeare, Othello, II, 3
IREG Forum University Rankings, Warsaw May 2013

3. IREG Forum University Rankings, Warsaw 16 -17 May 2013
In Global university rankings and their impact. Report II, EUA Report on rankings 2013
2.3 Superficial descriptions of methodology and poor indicators
…. Use of poor indicators also persists. In spite of widespread criticisms, reliance on reputation is becoming more and more widespread…..
IREG Forum University Rankings, Warsaw May 2013

4. Facts vs reputation Rankings graphical comparison as an approach
for deconvoluting ranking patterns
Reputation surveys:
ARWU: 0%, THE: 34.5%, QS: 40%
Webometrics as a proxy for reputation
IREG Forum University Rankings, Warsaw May 2013

5. Methodology 1 Ternary diagrams
IREG Forum University Rankings, Warsaw May 2013

6. Methodology 2 Ranking A vs Ranking B
IREG Forum University Rankings, Warsaw May 2013

7. Methodology 3 Absolute and relative differences
Diagrams
(ARWU-Y) vs ARWU
and
(ARWU-Y)/Y
vs ARWU
IREG Forum University Rankings, Warsaw May 2013

8. Methodology 4 Diagram ARWU/X vs ARWU
IREG Forum University Rankings, Warsaw May 2013

9. Methodology 5 Rankings reproducibilities
- years selected:
- average deviations to the average from one year to next:
ARWU: 2-3 %
THE: %
QS: 6-7 %
Webometrics: %
IREG Forum University Rankings, Warsaw May 2013

10. Methodology 6 Filtering rules
90 % of the universities (259) selected
62 % ranked in 4 rankings
6 % in 3 rankings
22 % in 2 rankings
First 250 ARWU ranks:
131 universities
IREG Forum University Rankings, Warsaw May 2013

11. IREG Forum University Rankings, Warsaw 16 -17 May 2013
Targets
United States
United Kingdom
Northern Europe
(Germany, Scandinavia, Netherlands)
Southern Europe
(France, Spain, Portugal, Italy)
IREG Forum University Rankings, Warsaw May 2013

12. Results 1 Ternary diagrams
IREG Forum University Rankings, Warsaw May 2013

13. Results 2 Ternary diagrams
IREG Forum University Rankings, Warsaw May 2013

14. IREG Forum University Rankings, Warsaw 16 -17 May 2013
Results 3 ARWU vs THE 1
IREG Forum University Rankings, Warsaw May 2013

15. IREG Forum University Rankings, Warsaw 16 -17 May 2013
Results 4 ARWU vs THE 2
IREG Forum University Rankings, Warsaw May 2013

16. IREG Forum University Rankings, Warsaw 16 -17 May 2013
Results 5 ARWU vs QS 1
IREG Forum University Rankings, Warsaw May 2013

17. IREG Forum University Rankings, Warsaw 16 -17 May 2013
Results 6 ARWU vs QS 2
IREG Forum University Rankings, Warsaw May 2013

18. Results 7 ARWU vs Webometrics 1
IREG Forum University Rankings, Warsaw May 2013

19. Results 8 ARWU vs Webometrics 2
IREG Forum University Rankings, Warsaw May 2013

20. Results 9 US: Relative differences
IREG Forum University Rankings, Warsaw May 2013

21. IREG Forum University Rankings, Warsaw 16 -17 May 2013
Results 10 Europe: Relative differences THE-QS: Continental Europe Webometrics=whole Europe
IREG Forum University Rankings, Warsaw May 2013

22. IREG Forum University Rankings, Warsaw 16 -17 May 2013
Summary
Underestimatation of universities best ranked for research
Overestimation of universities less well ranked
for research
Clear for THE and QS, even clearer for Webometrics
And for the best research universities:
US: underestimation THE/QS, overestimation Webometrics
UK: overestimation THE/QS, underestimation Webometrics
N + S Europe: underestimation THE/QS/Webometrics
IREG Forum University Rankings, Warsaw May 2013

23. IREG Forum University Rankings, Warsaw 16 -17 May 2013
Conclusions
Reputation concept still on its infancy
Necessity to do research on the concept as well as its modalities of implementation
IREG Forum University Rankings, Warsaw May 2013