14 March 2018 Briefing to the Higher Education Portfolio Committee on review of the draft 2018-19 APPs

Reputation promise/mission The Auditor-General of South Africa has a constitutional mandate and, as the Supreme Audit Institution (SAI) of South Africa, it exists to strengthen our country’s democracy by enabling oversight, accountability and governance in the public sector through auditing, thereby building public confidence.

Purpose of the briefing To provide the Portfolio Committee (PC) with audit insights on the interim review of 2018-19 draft annual performance plan (APPs) of the department in an effort to add value to oversight

Role of the AGSA in the review of the draft 2018-19 APP

Role of AGSA in the review of draft 2018-19 APP The review was performed on the final draft 2018-19 APP and all findings was communicated to the department to ensure that the APP submitted to the Portfolio Committee is according to the relevant frameworks. The AGSA reviewed the process followed for the preparation, submission and approval of strategic plans (if relevant) and APPs. Assessment of the measurability and relevance (excluding consistency and presentation) of the indicators and targets planned for each selected programme.

Background and applicable legislation

Background and applicable legislation National Development Plan (NDP) 2030 Medium Term Strategic Framework (MTSF) 2014-2019 Framework for Managing Programme Performance Information (FMPPI) Framework for Strategic Plans and Annual Performance Plans (APPs) National Treasury, Instruction Note No 33 - Framework for strategic and APPs

Key committee considerations when reviewing the APP Does the APP align with MTSF? Is there a logical link between objectives, indicators and targets? Does the APP include all programme performance indicators? Quarterly targets included for indicators that need to be reported quarterly? Are there technical indicator descriptions for indicators? Is each performance indicator well defined, verifiable AND relevant Is there clear and reliable baseline information for targets Is the overall alignment between the budget and APP evident and clear? Is each performance target specific, measurable AND time bound? Are the Strategic Plans and Annual Performance Plans aligned to the Medium-Term Strategic Framework (MTSF) outcomes and priorities? Are indicators relevant to the institution’s mandate and the realisation of strategic goals and objectives? Is there a logical link between objectives, indicators and targets? Does the APP include a complete set of indicators that is relevant to the objectives that need to be achieved e.g. are all customised sector indicators relevant to the institution included in the APP? For those programme performance indicators that need to be reported quarterly, are the quarterly targets included in the APP? For each programme performance indicator, does the technical indicator description contain minimum required information? Is each performance indicator well defined and verifiable, as defined in the framework? Is there a clear identification of other data sources used to determine the baseline of the performance target in cases where the target set for the year is new and prior year's performance can therefore not be used as a baseline? Is the overall alignment between the budget and APP evident and clear? Does each performance target comply with the SMART criteria?

AGSA review of the draft 2018-19 APP Review process Assessed the overall planning process followed by the department to prepare and submit strategic plans(if relevant) and APP. Reviewed draft annual performance plans (APP) for each selected programme, focusing on the same programmes that have been selected for the 2017-18 audit. Reporting Findings from the review are communicated in the 2017-18 interim management report to enable changes to be made. Findings relevant to the interim review do not have an impact on the audit conclusion of the selected programmes for the PFMA 2017-18 year end audit.

Criteria used to assess indicators A good performance indicator should be: Well-defined Verifiable Relevant Well-defined, i.e. clear, unambiguous definition so that data will be collected consistently and will be easy to understand and use. Verifiable, i.e. it must be possible to validate the processes and systems. Relevant, i.e Must relate logically and directly to an aspect of the SETA's mandate

S M A R T SMART criteria for performance targets PECIFIC EASURABLE Nature and required level of performance can be clearly identified PECIFIC M Required performance can be measured EASURABLE A Realistic given existing capacity CHIEVABLE R Required performance is linked to achievement of goal ELEVANT Time period/deadline for delivery is specified T IME BOUND For our review purposes, the focus was on: Specific i.e. the nature and the required level of performance can be clearly identified Measurable: the required performance can be measured Time bound: the time period or deadline for delivery is specified

Comparison between current year and prior year budget Note: The information contained in this slide must be populated with the programme and budget information relevant to specific auditees. This include an analysis of the budget, how much will be spent on various programmes, the purpose is to determine the extent to which the allocated resources are geared towards service delivery % Budget increase % Budget decrease

Budget analysis – Dept (cont.) Economic classification Department of Higher Education and Training % budget spend on employee remuneration Note: Please populate graph with information relevant to specific auditees.

Review findings on draft 2018/19 APP Department of Higher Education and Training (DHET) Scope We assessed the indicators and targets against the “SMART” criteria for programmes. Outcome The following indicators, from the selected programmes, were not well defined and did not have technical indicator descriptions : Programme 4: Technical and vocational education and training Programme 5: Skills development Programme 6: Community education and training

Review findings on draft 2018/19 APP Department of Higher Education and Training (DHET) Programme 4:Technical and vocational education and training Strategic Objective Performance indicator Annual Target Comment 4.4.1.2 To implement appropriate monitoring instruments for TVET colleges by 31 March 2021. 4.4.1.2.1 Number of reports on the implementation of steering mechanisms for TVET sector approved per annum 4 Implementation reports on steering mechanisms approved by the Director-General There is no technical indicator description provided Programme 5 : Skills development Strategic Objective Performance indicator Annual Target Comment 4.5.1.2 To implement appropriate monitoring instruments for the skills development system by 31 March 2021 4.5.1.2.1 Number of reports on the implementation of steering mechanisms for skills development approved 1 (100%)   A report on the implementation of the revised SETA landscape developed and approved by the Director-General by 31 March 2019 There is no technical indicator description provided furthermore the target is 1 but it also include the percentage

Review findings on draft 2018/19 APP Department of Higher Education and Training (DHET) Programme 6:Community education and training Strategic Objective Performance indicator Annual Target Comment 4.6.1.2 To implement appropriate monitoring instruments for the CET college sector by 31 March 2021 4.6.1.2.1 Number of reports on the implementation of steering mechanisms for the CET sector approved 1 (100%) A report on the implementation of Governance policies’ for Community colleges developed and approved by the Director-General by 31 March 2019 There is no technical indicator description provided furthermore the target is 1 but it also include the percentage Outcome/impact indicators Timeframe for reporting progress Head count enrolments in CET Colleges 320 000 Annually (3rd Quarter)(2017 academic year, reported and verified by 31 October 2018) No method of calculation was provided as a result the target may not be measureable. Source document to be used for collecting and reporting is not indicated Certification rates in CET formal qualifications (%) 40% Annually (3rd Quarter) (2017 academic year, reported and verified by 31 October 2018) No method of calculation provided as a result the target is not measureable.

Other matters Department of Higher Education and Training (DHET) Prior year finding Two indicators under programme 4 and 6 resulted in significant material findings in the 2016/17 Audit Report and actions plans to address these findings are still in process, there is a risk that these (Indicators) might results in repeat findings for 2017/18 and 2018/19 if not resolved Indicators affected Headcount enrolments in TVET colleges & CET Certification rates in TVET & CETs qualifications Findings Verification controls implemented by management over performance reporting were not fully effective to ensure reported information is valid, accurate and complete and supporting listings are accurate. This is as a result of: Lack of reporting regulations in place that requires TVET and CET colleges to report on headcount enrolments.

Review findings on draft 2018/19 APP Department of Higher Education and Training (DHET) Assessment of the progress made based on the action plan developed Although management has started with the implementation on certain verification processes for programme 4 and 6 during the year to implement recommendation made in the prior year, it should be noted that the planned processes have not been fully and effectively implemented by management. SOPs plans that have been developed have not been used to collate and report on performance information, the position of Data Quality Manager has not been filled. Below is an extract of activities indicated in the department’s action plan A Standard Operating Procedure (SoP) has been developed by TVET Student Development Support to ensure uniformity with regards to the NSFAS figures reported in the (Annual Performance Report(APR) at at 31 May each year. This SoP clearly indicate which data will be reported when, and how the NSFAS final figures will be incorporated when available. The SoP for verification of student enrolment has been approved by the DG on 14 September 2017, this documented procedures will be used for verification of information pending the finalisation of the policy. A fully documented plan has been developed on how the qualified audit outcome relating to enrolment data for TVET Colleges as reported in the DHET APR will be cleared. The plan address how TVETMIS will be fixed, how the Colleges will ensure that the required enrolment data records will be kept at TVET Colleges as well as the processes and procedures applied by the Chief Directorate to perform sample testing at the Colleges. Examinations is still to appoint a data quality manager that will develop and implement a testing protocol required in order to ensure that all data received from SITA is accurate, complete and valid.

SECTOR FOCUS AREAS

AGSA Service Offerings Regularity Audit Financial statements Compliance Performance information Information System Audit General IT Controls Data Analytics Investigation Performance Audit Economical Efficient Effective Fraud Detection Review Large projects

2014 – 2017 audit focus areas 2014-16 Value add areas - Research - Partnerships - Monitoring and Evaluation 2016-17 NSDS III Key Pillars - PIVOTAL/WIL - Rural Development - SMMEs/Unions - Inter-sectoral skills - Revitalization of TVET college 2017-18 Revitalization of TVET Colleges - SETA initiatives - NSF initiatives - DHET policy, guidance and support - NSFAS bursaries (WC

Rational for TVET as key focus for 2017-18 SDG Goal 4 – access to TVET programmes Increase the number of youth with skills that increases their employability chances and entrepreneurship NDP By 2030 the TVET sector should cover 25% of the age relevant cohort Targeted an increase from 300 000 to 1 250 000 by 2030 WP-PSET Prioritise strengthening and expanding of TVET colleges Make TVET colleges institutions of choice for school leavers NSDS III Promote the growth of TVET college system that is responsive to sector, local and national skills needs and priorities Struggling TVET colleges not receiving support Funds not allocated based on the needs and priorities of colleges Skills projects concentrated at selected TVET colleges

2017 – 18 Audit Theme: Revitalisation of the TVET Colleges Focus area Sub-focus areas 2017-18 SETA TVET college funded projects – teaching/learning facilities, skills interventions, lecturer development, management and leadership support, curriculum enhancement, etc. Rollout (PA) Rollout(PA) NSF DHET Planning, coordination and monitoring of TVET colleges revitalisation programme Staff turnover – race, gender, disability, age and area of specialisation NSFAS TVET college bursary administration The work will include follow-up on 2016-17 findings to have an updated research

Projects for revitalisation of TVET Colleges Bursaries Learnership Internship Skills Programmes Lecturer development Educational facilities Infrastructure refurbishment Curriculum enhancements Management and leadership development/support Any other TVET college project funded

SETA role in the revitalisation of TVET colleges The objective of the value-add audit is to determine whether: SETA prioritised the revitalisation of TVET colleges by initiating and providing financial support for programmes intended to improve and strengthen the sector. The funded interventions have been implemented as intended and whether they are aligned to the needs of the TVET colleges The audit approach include: Request for information, Review of the information received, Sampling of the funded TVET colleges to be visited by the audit team, Communication of findings for management comments, Detailed technical report, management paragraph, and Compilation of the skills development section in the education sector report (What we audited, what we found and what we recommend) The time line for the audit is February – June 2018 which include engagement of the draft paragraph for management letter. Confirmation will be provided on the scope in terms of projects active in 2017-18 or how many years back (3 years back for example)

QUESTIONS