Measuring differentiation in knowledge production at SA universities Johann Mouton, CREST HESA Conference on Research and Innovation 4 April 2012 Measuring differentiation in knowledge production at SA universities

Preliminary comments Given the different institutional histories, missions and capacities, a high degree of differentiation in terms of key research production dimensions are only to be expected The differentiation constructs and associated indicators presented and discussed here are not independent of each other (in statistical terms there are multiple “interaction effects”)

Unpacking the concept of “research differentiation” We still need a proper conceptualisation of the notion of research differentiation. As a first attempt I would distinguish the following SIX types or categories: Differentiation in terms of Volume of research production Shape of research production (differences in distribution of output by scientific field) Site of publication (comparative journal indexes) Research collaboration Research impact (High or low visibility or recognition) Demographics: Differences in distribution of output by gender/ race/ qualification/ age

Research differentiation indicators Dimension Indicators Volume Absolute nr of papers in peer-review journals (Institutional level) Normalized output (Nr of papers in peer reviewed journals divided by size of permanent academic staff – Institutional level) Shape Total nr of papers by scientific domain/field Site of publication Total nr of papers by journal index (ISI, ISI-SA, IBSS, SA, Scopus) Collaboration Nr of single institution papers Nr of nationally co-authored papers Nr of internationally co-authored papers Impact Journal normalized citation score (Institution level) Field-normalized citation score (Institutional level) Demographics Nr of papers by demographic category (gender, race, age intervals, highest qualification)

Proposition 1 University research production - since the introduction of a national research subsidy scheme in 1987 – initially remained quite stable (ranging between 5000 and 5500 article units between 1988 and 2003) BUT then increased dramatically to reach more than 8000 units in 2010. The best explanation for this dramatic increase is the introduction of the new research funding framework in 2003 (and which came into effect in 2005) which provided much more significant financial reward for research units and clearly provided a huge incentive to institutions to increase their output

Output of article equivalents: 1987 - 2009

Proposition 2 But the increase in recent years in absolute output has not affected the institutional distribution. The huge differences between the most productive and least productive universities that were evident 25 years ago, have remained mostly unchanged. A few universities have managed to improve their position in the ranking (UWC is a good example), but the vast inequalities in knowledge production between the top and bottom universities have not disappeared.

Total research publications output (1990 – 2009) INSTITUTION Total nr of Research Publication Equivalents Column % University of Pretoria 17981.52 14.6% University of KwaZulu-Natal 16246.66 13.2% University of Cape Town 15757.88 12.8% University of the Witwatersrand 15595.61 12.6% University of Stellenbosch 13251.65 10.7% University of South Africa 8786.84 7.1% University of the Free State 6938.24 5.6% University of Johannesburg 6264.10 5.1% North-West University 5268.50 4.3% Rhodes University 4306.78 3.5% University of the Western Cape 2895.35 2.3% Nelson Mandela Metropolitan University 2742.50 2.2% University of Limpopo 2017.29 0.8% Tshwane University of Technology 1037.38 University of Zululand 1017.38 University of Fort Hare 939.04 Cape Peninsula University of Technology 695.07 0.6% Walter Sisulu University 465.26 Durban University of Technology 415.65 0.3% Central University of Technology 317.28 University of Venda 225.81 0.2% Vaal University of Technology 177.72 0.1% Mangosuthu Technikon 35.21 0.0%

Total research publications output (1990 – 2009) – The top five INSTITUTION Total nr of Research Publication Equivalents Column % University of Pretoria 17981.52 14.6% University of KwaZulu-Natal 16246.66 13.2% University of Cape Town 15757.88 12.8% University of the Witwatersrand 15595.61 12.6% University of Stellenbosch 13251.65 10.7% THE TOP FIVE 78833.32 63.9% Rule: Universities producing more than 10% of total university output

Total research publications output (1990 – 2009) – The middle seven INSTITUTION Total nr of Research Publication Equivalents Column % University of South Africa 8786.84 7.1% University of the Free State 6938.24 5.6% University of Johannesburg 6264.10 5.1% North-West University 5268.50 4.3% Rhodes University 4306.78 3.5% University of the Western Cape 2895.35 2.3% Nelson Mandela Metropolitan University 2742.50 2.2% 37202.31 30.2% Rule: Universities poducing at least 1% of total sector output

Total research publications output (1990 – 2009) – The bottom eleven INSTITUTION Total nr of Research Publication Equivalents Column % University of Limpopo 2017.29 0.8% Tshwane University of Technology 1037.38 University of Zululand 1017.38 University of Fort Hare 939.04 Cape Peninsula University of Technology 695.07 0.6% Walter Sisulu University 465.26 Durban University of Technology 415.65 0.3% Central University of Technology 317.28 University of Venda 225.81 0.2% Vaal University of Technology 177.72 0.1% Mangosuthu Technikon 35.21 0.0%   7343.09 5.3%

Total DHET research output: 1990 - 2009

Highest versus lowest output institutions

A comment on institutional differentiation The statistics presented thus far on institutional output only refer to absolute output and have not been normalized for the size (viz. Academic capacity) of institutions. In the following two graphs we first present the rankings i.t.o. research output (normalized for number of permanent staff) and then the rankings i.t.o. knowledge output (Masters and Doctoral graduates included) also normalized for size of academic staff. A comparison of the two ranking reveal some interesting shifts in rankings (most notably for NMMU, UNISA and some of the UoT’s) but the overall difference in normalized output between the top and the bottom universities remains huge.

Ranking of universities i.t.o. per capita research output (2009)   Headcount of permanent staff Research Publication Units Accrued Per Capita Output UCT 982 1 253.03 1.28 2. US 917 1 034.70 1.13 3. RU 321 325.33 1.01 4. WITS 997 936.14 0.94 5. UKZN 1 403 1 146.51 0.82 6. UP 1 676 1 187.46 0.71 7. UJ 884 610.90 0.69 8. UFS 795 496.49 0.62 9. NWU 1 086 585.94 0.54 10. UWC 509 266.82 0.52 11. UNISA 1 404 734.60 12. UFH 292 142.22 0.49 13. NMMU 574 255.51 0.45 14. UZ 253 66.66 0.26 15. UV 76.76 0.24 16. TUT 820 188.06 0.23 17. CPUT 749 155.26 0.21 18. CUT 260 39.56 0.15 19. VUT 322 44.73 0.14 20. UL 770 93.25 0.12 21. WSU 608 51.85 0.09 22. DUT 48.45 0.08 23. MUT 152 7.57 0.05

Ranking of universities according to average normed knowledge production (2007 – 2009)  University Average annual weighted output 2007 – 2009 Average annual normed output for 2007 – 2009 1 (2) STELLENBOSCH 1833 177 2 (1) CAPE TOWN 1926 166 3 (3) RHODES 550 140 4 (4) WITWATERSRAND 1609 131 5 (6) PRETORIA 2216 110 6 (7) JOHANNESBURG 847 107 7 (5) KWA-ZULU NATAL 1768 103 8 (13) NELSON MANDELA 482 99 9 (9) NORTH WEST 1110 94 10 (8) FREE STATE 898 11 (10) WESTERN CAPE 505 82 12 (16) TSHWANE UT 277 75 13 (18) CENTRAL UT 74 61 14 (11) SOUTH AFRICA 938 60 15 (17) CAPE PENINSULA UT 184 16 (12) FORT HARE 199 53 17 (14) ZULULAND 146 49 18 (19) VAAL UT 43 33 19 (22) DURBAN UT 30 20 (20) LIMPOPO 243 25 21 (15) VENDA 80 24 22 (22) WALTER SISULU 6 23 (23) MANGOSUTHU 2 4

Proposition 3 SA universities vary hugely in terms of the “shape” of their knowledge production. The big differences in scientific field profiles of the different universities is clearly a function of institutional histories (e.g. having a medical school or faculty of theology) and institutional missions (research intensive universities versus more teaching universities and ex-technikons)

Shape of knowledge production (1990 -2005)

Proposition 4 Distribution of research output by journal index (ISI, IBSS and “SA”) varies hugely. The differences between the universities in terms of this dimension is mainly a function of the shape of knowledge production at the universities, but clearly also of other factors like institutional histories, language of publishing, and so on. One of the immediate results of these differences is its impact on university rankings.

Distribution of university output by journal index (1990 – 2005)

Proposition 4 University research output has become significantly more “international” and “collaborative” over the past 10 – 15 years. South African academics collaborate much more than before – in the post-apartheid sanction period this was always to be expected. But we also collaborate more in fields (such as infectious diseases) with international teams receiving huge international funding. Interestingly, there is nothing in the funding framework that actively encourages collaborative research – on the contrary. But one has to immediately add that this “negative” feature of the framework is offset by the positive effects of collaborative publishing as demonstrated in higher citations and more visibility.

Co-authorship trends by university for the period 1996 to 2007 (ISI-papers only) 97 98 99 00 01 02 03 04 05 06 07 Total Col % UWC SI 22 27 28 29 23 31 33 36 26 41 39 381 31.4% NC 14 19 25 18 30 38 35 295 24.3% IC 20 37 48 53 66 94 82 536 44.2% UCT 316 277 274 292 242 261 259 233 224 276 289 285 3529 34.5% 195 181 171 168 160 157 175 163 212 262 246 2522 24.7% 176 214 196 265 260 299 286 324 388 462 546 588 4168 40.8% NWU 43 32 40 44 46 57 484 33.2% 34 379 26.0% 21 63 64 91 72 74 593 40.7% UP 220 190 186 204 197 199 229 255 251 2781 39.7% 111 119 144 135 154 145 184 156 221 2015 28.8% 83 109 121 166 179 205 268 287 291 2202 31.5% UNISA 47 399 49.1% 17 15 12 10 8 13 328 20.2% 16 266 30.7%

Proposition 5 The impact of SA’s research production has increased significantly over the past 15 years – mostly because of collaborative publishing (in high-impact journals) – and possibly also because of increased research in highly visible research areas. This is true at the country level, but with very different impact levels at the institutional level.

South African ISI publication output Source: Robert Tijssen (CWTS, Leiden University, Netherlands); CWTS WoS database

South African citation impact (ISI: 2000 – 2010) Source: Robert Tijssen (CWTS, Leiden University, Netherlands); CWTS WoS database

Trends in field-normalised citation impact (“top five”)

Trends in citation impact of other ‘second tier’ research active universities

Distribution of citation impact of across fields of science (2004-2007)

Enablers of productivity and impact

Explaining the differences in research production International trends – the “demands” created by international rankings National steering instruments: Revised funding scheme + Expanded SA presence in ISI + NRf Rating system which have led to …. Increase in research output Increase in ISI-production Institutional capacities (Merton and cumulative advantage theory) Institutional histories and structures Institutional strategies (overleaf)

Institutional enablers We have seen how the institutional differences in research productive capacity has remained pretty much unchanged for the past 20 years. But how have the most productive universities (the top 5 – 7) managed to increase their absolute output so much more than some of the weakest institutions? How have some universities managed to increase their international visibility and impact much more significantly than others? There are at least two plausible (complementary) explanations – both relates to the human capital base. The first is evidence that shows that the top universities are not necessarily more productive at the individual level – they simply manage to broaden the active research base within the institution (cf. next slide). The second is the very persuasive evidence that shows the very strong correlation between the proportion of doctorate capacity and per capita research output (cf. following slide)

Depth of the human knowledgebase (WITS, UCT and UKZN) Publication year Total WITS articles Number of WITS authors responsible for articles Total UCT articles Number of UCT authors responsible for articles Total UKZN articles Number of UKZN authors responsible for articles 1990 895 927 932 973 557 591 1991 1160 901 898 886 554 537 1992 1164 870 926 573 609 1993 1177 939 877 897 615 608 1994 1270 975 945 953 577 584 1995 1204 974 636 661 1996 1090 958 823 923 660 621 1997 1106 984 880 950 748 708 1998 1026 907 892 902 651 653 1999 1127 983 960 992 721 702 2000 1120 966 917 735 688 2001 1078 930 981 744 716 2002 1077 961 1018 799 745 2003 947 750 842 851 985 725 2004 911 809 1047 1186 1023 717 2005 1070 1155 1140 726 2006 1166 1211 1086 632 2007 1242 1205

Comparison of WITS, UCT and UKZN i.t.o. research productivity Percentile breakdown of authors Nr of authors Mean nr of article equivalents WITS UCT UKZN 91-100% (Top 10% of authors) 462 652 411 13.66 11.41 13.61 71-90% 939 1302 823 3.38 2.40 3.22 51-70% 931 1.26 0.86 1.13 31-50% 937 0.59 0.43 0.56 11-30% 857 0.32 0.25 0.36 1-10% (Bottom 10% of authors) 542 0.16 0.12 0.19 Total 4668 6512 4114 2.48 1.95 2.43

Productivity (average nr of papers per permanent academic staff) and the % of permanent academic staff with PhDs, by individual university and the total headcount of permanent academic staff

In conclusion We undoubtedly have a highly differentiated university sector when assessed in terms of key and relevant indicators Some of the “causes” of these differences reflect the path-dependency of historical factors, missions and structures. Other differences are the results of more recent institutional responses to international and national policies, strategies and incentives. I have argued that the trends presented show that there are identifiable enabling mechanisms and drivers that impact on greater productivity and international impact even within a differentiated system.

The end