1. A view from the top- wishes, lies and statistics
SAAIR IR Foundations
12 September 2018

2. M&E
The main function of the M&E Directorate within CHE is to monitor the HE sector and developments at systemic level, and to carry out research to inform the CHE’s advisory function.
We rely on data to inform our work, especially HEMIS data. We think it’s important that policy development is based on rational analysis.

3. Activities of Directorate
Monitoring – data, newspapers, articles
Research – trends and developments
Draft advice – mainly for Minister of HE & T.

4. Scope of advice Areas of advice as contemplated in the HE Act:
qualifications, quality promotion and quality assurance (NQF Act)
research
the structure of the higher education system
the planning of the higher education system
a mechanism for the allocation of public funds
student financial aid
student support services
governance of higher education institutions and the system
language policy

5. Advice 2013-2018 POLICY COMMENT and ADVICE
Preliminary comment on the Draft National Plan on Post-School Education and Training (NPPSET) - March 2018
Preliminary comments on the Proposal for the National Skills Development Plan (NSDP) – February 2018
Preliminary comment on the draft Post-School Education and Training Information Policy – February 2018
Comment on the Data Confidentiality Standard – January 2018
Comment on the Data Dissemination Standard – September 2017
Advice on the proposed Policy on the Minimum Admission Requirements Requiring a National Senior Certificate for Adults (NASCA) – June 2017
Preliminary comment on the Draft Policy Framework for the Internationalisation of Higher Education in South Africa – June 2017
Comment on the Draft Policy Framework on Open Learning And Distance Education in South African Post-School Education and Training – March 2017
Advice on the Draft Policy on the Evaluation Of Creative Outputs and Innovations Produced by South African Public Higher Education Institutions – January 2017

6. ADVICE Contd. POLICY COMMENT & ADVICE
Comment on the Draft Policy Framework on Open Learning And Distance Education in South African Post-School Education and Training – March 2017
Advice on the Draft Policy on the Evaluation Of Creative Outputs and Innovations Produced by South African Public Higher Education Institutions – January 2017
Comment on the Policy for the Post-School Education and Training Central Applications Service – December 2016
Advice on Policy on Minimum Requirements for Programmes leading to Qualifications in Higher Education for Early Childhood Educators – December 2016
Comment on Draft Concept Note for the Macro-Indicator Report on Post-School Education and Training – December 2016
Comment on the Amended Regulations for the Registration of Private Higher Education Institutions – April 2016
Submission on the Proposal for the new National Skills Development Strategy (NSDS) and Sector Education and Training Authorities (SETAs) landscape – February 2016
Comment on the Draft Policy Framework on Social Inclusion – June 2015
Advice on Unaccredited programmes offered by public and private higher education institutions – December 2014
Advice on the Proposed improvements to the DHET’s policy and procedures for the measurement of research outputs of public higher education institutions - June 2014
Advice on the Policy for the Provision of Distance education in South African Universities in the context of an integrated post-school system – February 2014
Advice on Higher Education Tuition Services – February 2014
Comment on the Draft Regulations for the Establishment of a National Institute for the Humanities and Social Sciences – July 2013
Comment on the Proposed Policy on Student Housing at Public Universities – June 2013
Advice on the Status and Location of Public Colleges – March 2013

7. RESEARCH-BASED ADVICE
Advice on A Framework for the Regulation of Fees in Higher Education – April 2017
Advice on Fee Adjustments at Public Universities for 2017 – August 2016
Advice on the Reform of the Undergraduate Curriculum in South Africa (based on A Proposal for Undergraduate Curriculum Reform in South Africa: the case for a flexible curriculum structure) – December 2014  
Cohort studies

8. PUBLICATIONS SINCE 2013

9. Data-related work We do:
HEQCIS – system for private providers’ enrolment and achievement data
VitalStats – cohort studies
Institutional profiles – (new) – collaboration with IR
Data queries
Website info – maps etc.
BrieflySpeaking – interpretation e.g. national plan targets.

10. Scenario 1 – Missing teeth?

13. Quality of data input Importance of communicating what you want
Purpose of data
Importance of definitions
EXAMPLE:
What is an ‘enrolment’?
2 examples: - FTE definition and TVET numbers

14. What are we counting?
“Another change in methodology is the difference in reporting of headcount, a definition which has now been finalised. Headcount or distinct headcount reflects a student record count where a student is counted only once in an enrolment period/cycle, irrespective of the programme and subject enrolment and taking into account the Technical and Vocational Education and Training Colleges college the student is registered at. There are six (6) enrolment intakes/cycles at TVET colleges during an academic year – i.e. annual, trimester (3) and semester (2). Enrolment into the NC(V) qualification is annually, for Report 191 Engineering Studies enrolment is in trimesters and for the Business and General Studies programmes of Report 191, the intake is per semester. Previous statistical data on TVET colleges reported on programme headcount, which reflects the total number of enrolments for the enrolment cycle, irrespective of the unique student ID. Thus all programme and subject enrolments are counted, whether the same student is counted more than once during that enrolment intake/cycle.”

15. Scenario 2 – madness or method?
Differences sometimes relate to methodology. Depends on the assumptions made – different questions.
Examples: what’s a cohort? what’s in the basket? Which people, which qualifications, which mode, how long is regulation time, in which year do they graduate?

16. DHET cohort methodology
Cohort studies are the study of first time entering undergraduate students, who are tracked over a 10 year period to determine the percentage of students that have dropped out from their studies or who have completed their studies. The purpose of extending the study over a 10 year period is to take cognisance of the distance education method of educational provisioning.
Records are extracted from the HEMIS database for the base year data and filtered to only render the first-time entering undergraduate students. This includes students enrolled for three and four year undergraduate programmes. Only South African citizens are tracked, all the records containing non-valid South African National Identity numbers are removed from the dataset. The South African Identity number is used to track the progress of students.
The data for the base year consist of data fields for race, gender, field of study, graduation status, qualification type and the South African Identity number. Subsequent years do not need all these fields and only includes graduation status, qualification type and South African Identity number. It is assumed that the other fields remain the same throughout the study.

17. Vitalstats methodology

20. Differences Level of aggregation – per qualification types or all?
Foreign students included or not?
Mode – both contact and distance or contact only?
Is a change of qualification a dropout for that qualification or graduate in another?
ANSWER DIFFERENT QUESTIONS!

21. Scenario 3 – patterns of deception or truth?

22. EExplanation…
One provider accounts for a large proportion of the MBAs, which is the main masters type in private higher education.  Their numbers on their old MBA in 2013 were 1365, and in 2014 they were Then in 2015 they spiked to 4531, and there were no enrolments on that programme in 2016 or thereafter. In 2017, there were only 114 on their new MBA, and in 2018 there were On the previous Higher Education Qualifications Framework, the MBA was at level 8, and the admissions requirements in terms of previous qualifications were less stringent. It seems that the institution stopped offering their level 8 MBA in 2015, and introduced (as per the new regulations) a Postgraduate Diploma as a prerequisite for admission to the new MBA, which is now on level 9. In 2016, therefore, they registered most first-time entry students into the PGDip programme and not directly into the MBA, and as that is not a Masters degree, those numbers don’t reflect in the ISCED 7 table (we can find them under the PG Dips though).

23. Final note Working with data is fascinating.
There are all sorts of complicating factors:
Purpose
Assumptions
Methodology
Data capture or input
Filtering
Presentation
All of these can influence how one understands a phenomenon; they really do help in aiding understanding though!