1. 14 March 2018
Briefing to the Higher Education Portfolio Committee on review of the draft APPs

2. Reputation promise/mission
The Auditor-General of South Africa has a constitutional mandate and, as the Supreme Audit Institution (SAI) of South Africa, it exists to strengthen our country’s democracy by enabling oversight, accountability and governance in the public sector through auditing, thereby building public confidence.

3. Purpose of the briefing
To provide the Portfolio Committee (PC) with audit insights on the interim review of draft annual performance plan (APPs) of the department in an effort to add value to oversight

4. Role of the AGSA in the review of the draft 2018-19 APP

5. Role of AGSA in the review of draft 2018-19 APP
The review was performed on the final draft APP and all findings was communicated to the department to ensure that the APP submitted to the Portfolio Committee is according to the relevant frameworks.
The AGSA reviewed the process followed for the preparation, submission and approval of strategic plans (if relevant) and APPs.
Assessment of the measurability and relevance (excluding consistency and presentation) of the indicators and targets planned for each selected programme.

6. Background and applicable legislation

7. Background and applicable legislation
National Development Plan (NDP) 2030
Medium Term Strategic Framework (MTSF)
Framework for Managing Programme Performance Information (FMPPI)
Framework for Strategic Plans and Annual Performance Plans (APPs)
National Treasury, Instruction Note No Framework for strategic and APPs

8. Key committee considerations when reviewing the APP
Does the APP align with MTSF?
Is there a logical link between objectives, indicators and targets?
Does the APP include all programme performance indicators?
Quarterly targets included for indicators that need to be reported quarterly?
Are there technical indicator descriptions for indicators?
Is each performance indicator well defined, verifiable AND relevant
Is there clear and reliable baseline information for targets
Is the overall alignment between the budget and APP evident and clear?
Is each performance target specific, measurable AND time bound?
Are the Strategic Plans and Annual Performance Plans aligned to the Medium-Term Strategic Framework (MTSF) outcomes and priorities?
Are indicators relevant to the institution’s mandate and the realisation of strategic goals and objectives?
Is there a logical link between objectives, indicators and targets?
Does the APP include a complete set of indicators that is relevant to the objectives that need to be achieved e.g. are all customised sector indicators relevant to the institution included in the APP?
For those programme performance indicators that need to be reported quarterly, are the quarterly targets included in the APP?
For each programme performance indicator, does the technical indicator description contain minimum required information?
Is each performance indicator well defined and verifiable, as defined in the framework?
Is there a clear identification of other data sources used to determine the baseline of the performance target in cases where the target set for the year is new and prior year's performance can therefore not be used as a baseline?
Is the overall alignment between the budget and APP evident and clear?
Does each performance target comply with the SMART criteria?

9. AGSA review of the draft 2018-19 APP
Review process
Assessed the overall planning process followed by the department to prepare and submit strategic plans(if relevant) and APP.
Reviewed draft annual performance plans (APP) for each selected programme, focusing on the same programmes that have been selected for the audit.
Reporting
Findings from the review are communicated in the interim management report to enable changes to be made.
Findings relevant to the interim review do not have an impact on the audit conclusion of the selected programmes for the PFMA year end audit.

10. Criteria used to assess indicators
A good performance indicator should be:
Well-defined
Verifiable
Relevant
Well-defined, i.e. clear, unambiguous definition so that data will be collected consistently and will be easy to understand and use.
Verifiable, i.e. it must be possible to validate the processes and systems.
Relevant, i.e Must relate logically and directly to an aspect of the SETA's mandate

11. S M A R T SMART criteria for performance targets PECIFIC EASURABLE
Nature and required level of performance can be clearly identified
PECIFIC M
Required performance can be measured
EASURABLE A
Realistic given existing capacity
CHIEVABLE R
Required performance is linked to achievement of goal
ELEVANT
Time period/deadline for delivery is specified T
IME BOUND
For our review purposes, the focus was on:
Specific i.e. the nature and the required level of performance can be clearly identified
Measurable: the required performance can be measured
Time bound: the time period or deadline for delivery is specified

12. Comparison between current year and prior year budget
Note: The information contained in this slide must be populated with the programme and budget information relevant to specific auditees.
This include an analysis of the budget, how much will be spent on various programmes, the purpose is to determine the extent to which the allocated resources are geared towards service delivery
% Budget increase
% Budget decrease

13. Budget analysis – Dept (cont.)
Economic classification
Department of Higher Education and Training
% budget spend on employee remuneration
Note: Please populate graph with information relevant to specific auditees.

14. Review findings on draft 2018/19 APP
Department of Higher Education and Training (DHET)
Scope
We assessed the indicators and targets against the “SMART” criteria for programmes.
Outcome
The following indicators, from the selected programmes, were not well defined and did not have technical indicator descriptions :
Programme 4: Technical and vocational education and training
Programme 5: Skills development
Programme 6: Community education and training

15. Review findings on draft 2018/19 APP
Department of Higher Education and Training (DHET)
Programme 4:Technical and vocational education and training
Strategic Objective
Performance indicator
Annual Target
Comment
To implement appropriate monitoring instruments for TVET colleges by 31 March 2021.
Number of reports on the implementation of steering mechanisms for TVET sector approved per annum
4 Implementation reports on steering mechanisms approved by the Director-General
There is no technical indicator description provided
Programme 5 : Skills development
Strategic Objective
Performance indicator
Annual Target
Comment
To implement appropriate monitoring instruments for the skills development system by 31 March 2021
Number of reports on the implementation of steering mechanisms for skills development approved
1 (100%)
A report on the implementation of the revised SETA landscape developed and approved by the Director-General by 31 March 2019
There is no technical indicator description provided furthermore the target is 1 but it also include the percentage

16. Review findings on draft 2018/19 APP
Department of Higher Education and Training (DHET)
Programme 6:Community education and training
Strategic Objective
Performance indicator
Annual Target
Comment
To implement appropriate monitoring instruments for the CET college sector by 31 March 2021
Number of reports on the implementation of steering mechanisms for the CET sector approved
1 (100%)
A report on the implementation of Governance policies’ for Community colleges developed and approved by the Director-General by 31 March 2019
There is no technical indicator description provided furthermore the target is 1 but it also include the percentage
Outcome/impact indicators
Timeframe for reporting progress
Head count enrolments in CET Colleges
Annually (3rd Quarter)(2017 academic year, reported and verified by 31 October 2018)
No method of calculation was provided as a result the target may not be measureable. Source document to be used for collecting and reporting is not indicated
Certification rates in CET formal qualifications (%)
40%
Annually (3rd Quarter) (2017 academic year, reported and verified by 31 October 2018)
No method of calculation provided as a result the target is not measureable.

17. Other matters Department of Higher Education and Training (DHET)
Prior year finding
Two indicators under programme 4 and 6 resulted in significant material findings in the 2016/17 Audit Report and actions plans to address these findings are still in process, there is a risk that these (Indicators) might results in repeat findings for 2017/18 and 2018/19 if not resolved
Indicators affected
Headcount enrolments in TVET colleges & CET
Certification rates in TVET & CETs qualifications
Findings
Verification controls implemented by management over performance reporting were not fully effective to ensure reported information is valid, accurate and complete and supporting listings are accurate. This is as a result of:
Lack of reporting regulations in place that requires TVET and CET colleges to report on headcount enrolments.

18. Review findings on draft 2018/19 APP
Department of Higher Education and Training (DHET)
Assessment of the progress made based on the action plan developed
Although management has started with the implementation on certain verification processes for programme 4 and 6 during the year to implement recommendation made in the prior year, it should be noted that the planned processes have not been fully and effectively implemented by management.
SOPs plans that have been developed have not been used to collate and report on performance information,
the position of Data Quality Manager has not been filled.
Below is an extract of activities indicated in the department’s action plan
A Standard Operating Procedure (SoP) has been developed by TVET Student Development Support to ensure uniformity with regards to the NSFAS figures reported in the (Annual Performance Report(APR) at at 31 May each year. This SoP clearly indicate which data will be reported when, and how the NSFAS final figures will be incorporated when available.
The SoP for verification of student enrolment has been approved by the DG on 14 September 2017, this documented procedures will be used for verification of information pending the finalisation of the policy.
A fully documented plan has been developed on how the qualified audit outcome relating to enrolment data for TVET Colleges as reported in the DHET APR will be cleared. The plan address how TVETMIS will be fixed, how the Colleges will ensure that the required enrolment data records will be kept at TVET Colleges as well as the processes and procedures applied by the Chief Directorate to perform sample testing at the Colleges.
Examinations is still to appoint a data quality manager that will develop and implement a testing protocol required in order to ensure that all data received from SITA is accurate, complete and valid.

19. SECTOR FOCUS AREAS

20. AGSA Service Offerings
Regularity Audit
Financial statements
Compliance
Performance information
Information System Audit
General IT Controls
Data Analytics
Investigation
Performance Audit
Economical
Efficient
Effective
Fraud Detection
Review Large projects

21. 2014 – 2017 audit focus areas
Value add areas
- Research
- Partnerships
- Monitoring and Evaluation
NSDS III Key Pillars
- PIVOTAL/WIL
- Rural Development
- SMMEs/Unions
- Inter-sectoral skills
- Revitalization of TVET college
Revitalization of TVET Colleges
- SETA initiatives
- NSF initiatives
- DHET policy, guidance and support
- NSFAS bursaries (WC

22. Rational for TVET as key focus for 2017-18
SDG
Goal 4 – access to TVET programmes
Increase the number of youth with skills that increases their employability chances and entrepreneurship
NDP
By 2030 the TVET sector should cover 25% of the age relevant cohort
Targeted an increase from to by 2030
WP-PSET
Prioritise strengthening and expanding of TVET colleges
Make TVET colleges institutions of choice for school leavers
NSDS III
Promote the growth of TVET college system that is responsive to sector, local and national skills needs and priorities
Struggling TVET colleges not receiving support
Funds not allocated based on the needs and priorities of colleges
Skills projects concentrated at selected TVET colleges

23. 2017 – 18 Audit Theme: Revitalisation of the TVET Colleges
Focus area
Sub-focus areas
SETA
TVET college funded projects – teaching/learning facilities, skills interventions, lecturer development, management and leadership support, curriculum enhancement, etc.
Rollout (PA)
Rollout(PA)
NSF
DHET
Planning, coordination and monitoring of TVET colleges revitalisation programme
Staff turnover – race, gender, disability, age and area of specialisation
NSFAS
TVET college bursary administration
The work will include follow-up on findings to have an updated research

24. Projects for revitalisation of TVET Colleges
Bursaries
Learnership
Internship
Skills Programmes
Lecturer development
Educational facilities
Infrastructure refurbishment
Curriculum enhancements
Management and leadership development/support
Any other TVET college project funded

25. SETA role in the revitalisation of TVET colleges
The objective of the value-add audit is to determine whether:
SETA prioritised the revitalisation of TVET colleges by initiating and providing financial support for programmes intended to improve and strengthen the sector.
The funded interventions have been implemented as intended and whether they are aligned to the needs of the TVET colleges
The audit approach include:
Request for information,
Review of the information received,
Sampling of the funded TVET colleges to be visited by the audit team,
Communication of findings for management comments,
Detailed technical report,
management paragraph, and
Compilation of the skills development section in the education sector report (What we audited, what we found and what we recommend)
The time line for the audit is February – June 2018 which include engagement of the draft paragraph for management letter.
Confirmation will be provided on the scope in terms of projects active in or how many years back (3 years back for example)

26. QUESTIONS