1. Measuring differentiation in knowledge production at SA universities
Johann Mouton, CREST
HESA Conference on Research and Innovation
4 April 2012
Measuring differentiation in knowledge production at SA universities

2. Preliminary comments
Given the different institutional histories, missions and capacities, a high degree of differentiation in terms of key research production dimensions are only to be expected
The differentiation constructs and associated indicators presented and discussed here are not independent of each other (in statistical terms there are multiple “interaction effects”)

3. Unpacking the concept of “research differentiation”
We still need a proper conceptualisation of the notion of research differentiation. As a first attempt I would distinguish the following SIX types or categories:
Differentiation in terms of
Volume of research production
Shape of research production (differences in distribution of output by scientific field)
Site of publication (comparative journal indexes)
Research collaboration
Research impact (High or low visibility or recognition)
Demographics: Differences in distribution of output by gender/ race/ qualification/ age

4. Research differentiation indicators
Dimension
Indicators
Volume
Absolute nr of papers in peer-review journals (Institutional level)
Normalized output (Nr of papers in peer reviewed journals divided by size of permanent academic staff – Institutional level)
Shape
Total nr of papers by scientific domain/field
Site of publication
Total nr of papers by journal index (ISI, ISI-SA, IBSS, SA, Scopus)
Collaboration
Nr of single institution papers
Nr of nationally co-authored papers
Nr of internationally co-authored papers
Impact
Journal normalized citation score (Institution level)
Field-normalized citation score (Institutional level)
Demographics
Nr of papers by demographic category (gender, race, age intervals, highest qualification)

5. Proposition 1
University research production - since the introduction of a national research subsidy scheme in 1987 – initially remained quite stable (ranging between 5000 and 5500 article units between 1988 and 2003) BUT then increased dramatically to reach more than 8000 units in The best explanation for this dramatic increase is the introduction of the new research funding framework in 2003 (and which came into effect in 2005) which provided much more significant financial reward for research units and clearly provided a huge incentive to institutions to increase their output

6. Output of article equivalents: 1987 - 2009

7. Proposition 2
But the increase in recent years in absolute output has not affected the institutional distribution. The huge differences between the most productive and least productive universities that were evident 25 years ago, have remained mostly unchanged. A few universities have managed to improve their position in the ranking (UWC is a good example), but the vast inequalities in knowledge production between the top and bottom universities have not disappeared.

8. Total research publications output (1990 – 2009)
INSTITUTION
Total nr of Research Publication Equivalents
Column %
University of Pretoria
14.6%
University of KwaZulu-Natal
13.2%
University of Cape Town
12.8%
University of the Witwatersrand
12.6%
University of Stellenbosch
10.7%
University of South Africa
7.1%
University of the Free State
5.6%
University of Johannesburg
5.1%
North-West University
4.3%
Rhodes University
3.5%
University of the Western Cape
2.3%
Nelson Mandela Metropolitan University
2.2%
University of Limpopo
0.8%
Tshwane University of Technology
University of Zululand
University of Fort Hare
939.04
Cape Peninsula University of Technology
695.07
0.6%
Walter Sisulu University
465.26
Durban University of Technology
415.65
0.3%
Central University of Technology
317.28
University of Venda
225.81
0.2%
Vaal University of Technology
177.72
0.1%
Mangosuthu Technikon
35.21
0.0%

9. Total research publications output (1990 – 2009) – The top five
INSTITUTION
Total nr of Research Publication Equivalents
Column %
University of Pretoria
14.6%
University of KwaZulu-Natal
13.2%
University of Cape Town
12.8%
University of the Witwatersrand
12.6%
University of Stellenbosch
10.7%
THE TOP FIVE
63.9%
Rule: Universities producing more than 10% of total university output

10. Total research publications output (1990 – 2009) – The middle seven
INSTITUTION
Total nr of Research Publication Equivalents
Column %
University of South Africa
7.1%
University of the Free State
5.6%
University of Johannesburg
5.1%
North-West University
4.3%
Rhodes University
3.5%
University of the Western Cape
2.3%
Nelson Mandela Metropolitan University
2.2%
30.2%
Rule: Universities poducing at least 1% of total sector output

11. Total research publications output (1990 – 2009) – The bottom eleven
INSTITUTION
Total nr of Research Publication Equivalents
Column %
University of Limpopo
0.8%
Tshwane University of Technology
University of Zululand
University of Fort Hare
939.04
Cape Peninsula University of Technology
695.07
0.6%
Walter Sisulu University
465.26
Durban University of Technology
415.65
0.3%
Central University of Technology
317.28
University of Venda
225.81
0.2%
Vaal University of Technology
177.72
0.1%
Mangosuthu Technikon
35.21
0.0%
5.3%

12. Total DHET research output: 1990 - 2009

13. Highest versus lowest output institutions

14. A comment on institutional differentiation
The statistics presented thus far on institutional output only refer to absolute output and have not been normalized for the size (viz. Academic capacity) of institutions. In the following two graphs we first present the rankings i.t.o. research output (normalized for number of permanent staff) and then the rankings i.t.o. knowledge output (Masters and Doctoral graduates included) also normalized for size of academic staff. A comparison of the two ranking reveal some interesting shifts in rankings (most notably for NMMU, UNISA and some of the UoT’s) but the overall difference in normalized output between the top and the bottom universities remains huge.

15. Ranking of universities i.t.o. per capita research output (2009)
Headcount of permanent staff
Research Publication Units Accrued
Per Capita Output
UCT
982
1.28
2. US
917
1.13
3. RU
321
325.33
1.01
4. WITS
997
936.14
0.94
5. UKZN
1 403
0.82
6. UP
1 676
0.71
7. UJ
884
610.90
0.69
8. UFS
795
496.49
0.62
9. NWU
1 086
585.94
0.54
10. UWC
509
266.82
0.52
11. UNISA
1 404
734.60
12. UFH
292
142.22
0.49
13. NMMU
574
255.51
0.45
14. UZ
253
66.66
0.26
15. UV
76.76
0.24
16. TUT
820
188.06
0.23
17. CPUT
749
155.26
0.21
18. CUT
260
39.56
0.15
19. VUT
322
44.73
0.14
20. UL
770
93.25
0.12
21. WSU
608
51.85
0.09
22. DUT
48.45
0.08
23. MUT
152
7.57
0.05

16. Ranking of universities according to average normed knowledge production (2007 – 2009)
 University
Average annual weighted output 2007 – 2009
Average annual normed output for 2007 – 2009
1 (2)
STELLENBOSCH
1833
177
2 (1)
CAPE TOWN
1926
166
3 (3)
RHODES
550
140
4 (4)
WITWATERSRAND
1609
131
5 (6)
PRETORIA
2216
110
6 (7)
JOHANNESBURG
847
107
7 (5)
KWA-ZULU NATAL
1768
103
8 (13)
NELSON MANDELA
482 99
9 (9)
NORTH WEST
1110 94
10 (8)
FREE STATE
898
11 (10)
WESTERN CAPE
505 82
12 (16)
TSHWANE UT
277 75
13 (18)
CENTRAL UT 74 61
14 (11)
SOUTH AFRICA
938 60
15 (17)
CAPE PENINSULA UT
184
16 (12)
FORT HARE
199 53
17 (14)
ZULULAND
146 49
18 (19)
VAAL UT 43 33
19 (22)
DURBAN UT 30
20 (20)
LIMPOPO
243 25
21 (15)
VENDA 80 24
22 (22)
WALTER SISULU 6
23 (23)
MANGOSUTHU 2 4

17. Proposition 3
SA universities vary hugely in terms of the “shape” of their knowledge production. The big differences in scientific field profiles of the different universities is clearly a function of institutional histories (e.g. having a medical school or faculty of theology) and institutional missions (research intensive universities versus more teaching universities and ex-technikons)

18. Shape of knowledge production (1990 -2005)

19. Proposition 4
Distribution of research output by journal index (ISI, IBSS and “SA”) varies hugely. The differences between the universities in terms of this dimension is mainly a function of the shape of knowledge production at the universities, but clearly also of other factors like institutional histories, language of publishing, and so on. One of the immediate results of these differences is its impact on university rankings.

20. Distribution of university output by journal index (1990 – 2005)

21. Proposition 4
University research output has become significantly more “international” and “collaborative” over the past 10 – 15 years. South African academics collaborate much more than before – in the post-apartheid sanction period this was always to be expected. But we also collaborate more in fields (such as infectious diseases) with international teams receiving huge international funding. Interestingly, there is nothing in the funding framework that actively encourages collaborative research – on the contrary. But one has to immediately add that this “negative” feature of the framework is offset by the positive effects of collaborative publishing as demonstrated in higher citations and more visibility.

22. Co-authorship trends by university for the period 1996 to 2007 (ISI-papers only)97 98 99 00 01 02 03 04 05 06 07
Total
Col %
UWC SI 22 27 28 29 23 31 33 36 26 41 39
381
31.4% NC 14 19 25 18 30 38 35
295
24.3% IC 20 37 48 53 66 94 82
536
44.2%
UCT
316
277
274
292
242
261
259
233
224
276
289
285
3529
34.5%
195
181
171
168
160
157
175
163
212
262
246
2522
24.7%
176
214
196
265
260
299
286
324
388
462
546
588
4168
40.8%
NWU 43 32 40 44 46 57
484
33.2% 34
379
26.0% 21 63 64 91 72 74
593
40.7% UP
220
190
186
204
197
199
229
255
251
2781
39.7%
111
119
144
135
154
145
184
156
221
2015
28.8% 83
109
121
166
179
205
268
287
291
2202
31.5%
UNISA 47
399
49.1% 17 15 12 10 8 13
328
20.2% 16
266
30.7%

23. Proposition 5
The impact of SA’s research production has increased significantly over the past 15 years – mostly because of collaborative publishing (in high-impact journals) – and possibly also because of increased research in highly visible research areas. This is true at the country level, but with very different impact levels at the institutional level.

24. South African ISI publication output
Source: Robert Tijssen (CWTS, Leiden University, Netherlands); CWTS WoS database

25. South African citation impact (ISI: 2000 – 2010)
Source: Robert Tijssen (CWTS, Leiden University, Netherlands); CWTS WoS database

26. Trends in field-normalised citation impact (“top five”)

27. Trends in citation impact of other ‘second tier’ research active universities

28. Distribution of citation impact of across fields of science (2004-2007)

29. Enablers of productivity and impact

30. Explaining the differences in research production
International trends – the “demands” created by international rankings
National steering instruments: Revised funding scheme + Expanded SA presence in ISI + NRf Rating system which have led to ….
Increase in research output
Increase in ISI-production
Institutional capacities (Merton and cumulative advantage theory)
Institutional histories and structures
Institutional strategies (overleaf)

31. Institutional enablers
We have seen how the institutional differences in research productive capacity has remained pretty much unchanged for the past 20 years. But how have the most productive universities (the top 5 – 7) managed to increase their absolute output so much more than some of the weakest institutions? How have some universities managed to increase their international visibility and impact much more significantly than others?
There are at least two plausible (complementary) explanations – both relates to the human capital base. The first is evidence that shows that the top universities are not necessarily more productive at the individual level – they simply manage to broaden the active research base within the institution (cf. next slide). The second is the very persuasive evidence that shows the very strong correlation between the proportion of doctorate capacity and per capita research output (cf. following slide)

32. Depth of the human knowledgebase (WITS, UCT and UKZN)
Publication year
Total WITS articles
Number of WITS authors responsible for articles
Total UCT articles
Number of UCT authors responsible for articles
Total UKZN
articles
Number of UKZN authors responsible for articles
1990
895
927
932
973
557
591
1991
1160
901
898
886
554
537
1992
1164
870
926
573
609
1993
1177
939
877
897
615
608
1994
1270
975
945
953
577
584
1995
1204
974
636
661
1996
1090
958
823
923
660
621
1997
1106
984
880
950
748
708
1998
1026
907
892
902
651
653
1999
1127
983
960
992
721
702
2000
1120
966
917
735
688
2001
1078
930
981
744
716
2002
1077
961
1018
799
745
2003
947
750
842
851
985
725
2004
911
809
1047
1186
1023
717
2005
1070
1155
1140
726
2006
1166
1211
1086
632
2007
1242
1205

33. Comparison of WITS, UCT and UKZN i.t.o. research productivity
Percentile breakdown of authors
Nr of authors
Mean nr of article equivalents
WITS
UCT
UKZN
91-100% (Top 10% of authors)
462
652
411
13.66
11.41
13.61
71-90%
939
1302
823
3.38
2.40
3.22
51-70%
931
1.26
0.86
1.13
31-50%
937
0.59
0.43
0.56
11-30%
857
0.32
0.25
0.36
1-10% (Bottom 10% of authors)
542
0.16
0.12
0.19
Total
4668
6512
4114
2.48
1.95
2.43

34. Productivity (average nr of papers per permanent academic staff) and the % of permanent academic staff with PhDs, by individual university and the total headcount of permanent academic staff

35. In conclusion
We undoubtedly have a highly differentiated university sector when assessed in terms of key and relevant indicators
Some of the “causes” of these differences reflect the path-dependency of historical factors, missions and structures. Other differences are the results of more recent institutional responses to international and national policies, strategies and incentives.
I have argued that the trends presented show that there are identifiable enabling mechanisms and drivers that impact on greater productivity and international impact even within a differentiated system.

36. The end