15 March 2017 Briefing to Portfolio Committee of the Higher Education and Training on review of the draft 2017-18 APPs

Reputation promise/mission The Auditor-General of South Africa has a constitutional mandate and, as the Supreme Audit Institution (SAI) of South Africa, it exists to strengthen our country’s democracy by enabling oversight, accountability and governance in the public sector through auditing, thereby building public confidence.

Purpose of the briefing To provide the chairperson of the Portfolio Committee (PC) with audit insights on the interim review of 2017-18 draft annual performance plans (APPs) of the department and its public entities in an effort to add value to oversight

Key committee considerations when reviewing the APP Does the APP align with MTSF? Is there a logical link between objectives, indicators and targets? Does the APP include all programme performance indicators? Quarterly targets included for indicators that need to be reported quarterly? Are there technical indicator descriptions for indicators? Is each performance indicator well defined, verifiable AND relevant Is there clear and reliable baseline information for targets Is the overall alignment between the budget and APP evident and clear? Is each performance target specific, measurable AND time bound? Are the Strategic Plans and Annual Performance Plans aligned to the Medium-Term Strategic Framework (MTSF) outcomes and priorities? Are indicators relevant to the institution’s mandate and the realisation of strategic goals and objectives? Is there a logical link between objectives, indicators and targets? Does the APP include a complete set of indicators that is relevant to the objectives that need to be achieved e.g. are all customised sector indicators relevant to the institution included in the APP? For those programme performance indicators that need to be reported quarterly, are the quarterly targets included in the APP? For each programme performance indicator, does the technical indicator description contain minimum required information? Is each performance indicator well defined and verifiable, as defined in the framework? Is there a clear identification of other data sources used to determine the baseline of the performance target in cases where the target set for the year is new and prior year's performance can therefore not be used as a baseline? Is the overall alignment between the budget and APP evident and clear? Does each performance target comply with the SMART criteria?

Review process Reporting AGSA review of the draft 2017-18 APP Assessed the overall planning process followed by the Department and entities to prepare and submit strategic plans (if relevant) and APPs. Reviewed draft annual performance plans (APP) for each selected programme, focusing on the same programmes that have been selected for the 2016-17 audit. Reporting Findings from the review are communicated in the 2016-17 interim management report to enable changes to be made. Findings relevant to the interim review do not have an impact on the audit conclusion of the selected programmes for the PFMA 2016-17 year end audit.

Criteria used to assess indicators A good performance indicator should be: Well-defined Verifiable Relevant Well-defined, i.e. clear, unambiguous definition so that data will be collected consistently and will be easy to understand and use. Verifiable, i.e. it must be possible to validate the processes and systems. Relevant, i.e Must relate logically and directly to an aspect of the entity’s mandate

S M A R T SMART criteria for performance targets PECIFIC EASURABLE Nature and required level of performance can be clearly identified PECIFIC M Required performance can be measured EASURABLE A Realistic given existing capacity CHIEVABLE R Required performance is linked to achievement of goal ELEVANT Time period/deadline for delivery is specified T IME BOUND For our review purposes, the focus was on: Specific i.e. the nature and the required level of performance can be clearly identified Measurable: the required performance can be measured Time bound: the time period or deadline for delivery is specified

Comparison between current year and prior year budget Note: The information contained in this slide must be populated with the programme and budget information relevant to specific auditees. This include an analysis of the budget, how much will be spent on various programmes, the purpose is to determine the extent to which the allocated resources are geared towards service delivery % Budget increase % Budget decrease % %

Budget analysis – Dept (cont.) Economic classification Department of Higher Education and Training % budget spend on employee remuneration Note: Please populate graph with information relevant to specific auditees.

Review findings on draft 2017/18 APP Department of Higher Education and Training (DHET) Scope We assessed the indicators and targets against the “SMART” criteria for programmes. Outcome No significant findings were identified for the following selected programmes: Programme 2: Planning, policy and strategy Programme 3: University education Programme 4: Technical and vocational education and training Programme 5: Skills development Programme 6: Community education and training Other matters Three outcome indicators under programme 4 have zero targets due to no funding being available, as reported in the APP.

Review findings on draft 2017/18 APP Significant findings were identified for the following entities: Public entities Entity 2017/18 2016/17 FOODBEV Yes MERSETA SASSETA CHE

No. of indicators tested % of targets not specific Review findings on draft 2017/18 APP Public entities (cont.) Indicators not well-defined Entity No. of indicators tested Findings % of targets not specific FOODBEV 17 4 24% MERSETA 37 9 SASSETA 33 12% Use slide if interim review was performed and findings were raised. Specific Measurable Timebound Targets Well defined Verifiable Relevant Indicators

No. of indicators tested % of targets not specific Review findings on draft 2017/18 APP Public entities (cont.) Indicators not verifiable Entity No. of indicators tested Findings % of targets not specific FOODBEV 17 4 24% Use slide if interim review was performed and findings were raised. Specific Measurable Timebound Targets Well defined Verifiable Relevant Indicators

No. of indicators tested % of targets not specific Review findings on draft 2017/18 APP Public entities (cont.) Indicators not complete Entity No. of indicators tested Findings % of targets not specific SASSETA 11 5 45% Use slide if interim review was performed and findings were raised. Specific Measurable Timebound Targets Well defined Verifiable Relevant Indicators

% of targets not specific Review findings on draft 2017/18 APP Public entities (cont.) Targets not specific Entity No. of targets tested Findings % of targets not specific MERSETA 26 12 46% CHE 51 9 18% Use slide if interim review was performed and findings were raised. Specific Measurable Timebound Targets Well defined Verifiable Relevant Indicators

% of targets not specific Review findings on draft 2017/18 APP Public entities (cont.) Targets not measurable Entity No. of targets tested Findings % of targets not specific SASSETA 33 4 12% CHE 32 9 28% Use slide if interim review was performed and findings were raised. Specific Measurable Timebound Targets Well defined Verifiable Relevant Indicators

% of targets not specific No significant findings were identified Review findings on draft 2017/18 APP Public entities (cont.) Targets not time-bound Entity No. of targets tested Findings % of targets not specific No significant findings were identified Use slide if interim review was performed and findings were raised. Specific Measurable Timebound Targets Well defined Verifiable Relevant Indicators

QUESTIONS