1. 13 March 2018
Briefing to the Portfolio Committee on Basic Education: Review of the draft Annual Performance Plan of the Department of Basic Education

2. Reputation promise/mission
The Auditor-General of South Africa has a constitutional mandate and, as the Supreme Audit Institution (SAI) of South Africa, it exists to strengthen our country’s democracy by enabling oversight, accountability and governance in the public sector through auditing, thereby building public confidence.

3. Role of the AGSA in the review of the draft 2018-19 APP

4. 1. Role of AGSA in the review of draft 2018-19 APP
Our role as the AGSA is to reflect on the review performed of the draft 2018/19 APP to assist the portfolio committee in its oversight role in assessing the APP presented by the DBE to the portfolio committee taking into consideration the objective of the committee to the APP.
To enable oversight to focus on areas that will lead to good governance.
The review was performed on the final draft APP and all findings was communicated to the DBE to ensure that the APP submitted to the Portfolio Committee is according to the relevant frameworks.
The AG reviewed the process followed for the preparation, submission and approval of strategic plans (if relevant) and APPs.
Assessment of the measurability and relevance (excluding consistency and presentation) of the indicators and targets planned for each selected programme.

5. 1. Role of AGSA in the review of draft 2018-19 APP (Cont…)
NOTE: The findings relevant to the review of the draft APPs will not have an impact on the audit conclusion on usefulness or reliability of the selected programmes for the PFMA year end audit.
All AGSA findings have been communicated to DBE and DBE confirmed that they are in the process of taking our suggestions into consideration.
It should be noted that we are still awaiting the final APP to confirm that the confirmed corrections have been corrected.

6. Background and applicable legislation2

7. 2. Background and applicable legislation
National Development Plan (NDP) 2030
Medium Term Strategic Framework (MTSF)
Framework for Managing Programme Performance Information (FMPPI)
Framework for Strategic Plans and Annual Performance Plans (APPs)
National Treasury, Instruction Note No Framework for strategic and APPs

8. 2.1. Criteria used to assess indicators
A good performance indicator should be:
Well-defined
Verifiable
Relevant
Well-defined, i.e. clear, unambiguous definition so that data will be collected consistently and will be easy to understand and use.
Verifiable, i.e. it must be possible to validate the processes and systems.
Relevant, i.e Must relate logically and directly to an aspect of the SETA's mandate

9. S M A R T 2.2. SMART criteria for performance targets PECIFIC
Nature and required level of performance can be clearly identified
PECIFIC M
Required performance can be measured
EASURABLE A
Realistic given existing capacity
CHIEVABLE R
Required performance is linked to achievement of goal
ELEVANT
For our review purposes, the focus was on:
Specific i.e. the nature and the required level of performance can be clearly identified
Measurable: the required performance can be measured
Time bound: the time period or deadline for delivery is specified
Time period/deadline for delivery is specified T
IME BOUND

10. AGSA findings identified and communicated to DBE3

11. 3.1 Number of Usefulness findings per programme
DBE Programmes
Number of usefulness findings per programme
1. Administration
2. Curriculum Policy, Support and Monitoring 6
3. Teachers, Education Human Resources and Institutional Development 1
4. Planning, Information and Assessment 3
5. Educational Enrichment Services
AGSA findings identified and communicated to DBE 3

12. 3.2. AGSA Findings identified
In terms of the framework for strategic plans and annual performance, states that “each objective should be written as a performance statement that is SMART and must set a performance target the institution can achieve by the end of the period of the Strategic Plan. The baseline and targets must be expressed in terms of numbers. If a percentage is used, then the absolute numbers must be presented as well. Through our review of the draft annual performance plan, we noted that the targets that are expressed in percentages did not have the absolute values as required by the framework.
Indicators relating to HIV and Aids (Life Skills Education) DORA conditional grant are not included in the annual performance plan; (e.g)
Number of educators trained to implement Sexual and Reproductive Health (SRH) programmes for learners
Number of LTSM on Sexual Reproductive Health (SRH) distributed to schools.
Through the inspection of the annual performance plan for the year 2018/19 financial year, we noted that some of the indicator are indicated as “cumulative but the targets are not listed as cumulative under the MTSF period. Therefore, it is recommended that targets that are cumulative per quarter are clearly indicated in the TID.

13. 3.2. AGSA Findings identified (continued…)
DBE have been in discussions with PED’s throughout the year and have started standardising some of the MTSF indicators and including them as customised indicators for 2018/19.
PED has started aligning their annual performance plan to MTSF indicators but not all indicators are included.
Some MTSF indicator (ANA) cannot be implemented as yet by the PEDs. This is because DBE is remodelling Annual National Assessment (ANA). In this regard, key milestones are not included in the APP of the DBE and therefore MTSF indicators are not measured for performance within the education sector.
Performance measures in the NDP and MTSF are not translated into performance indicators resulting in poor alignment between the national and provincial departments of education (SMART principle).

14. 3.3. AGSA Findings identified (continued…)
Through review of the medium term strategic framework (MTSF) , referenced to outcome number 1 – Quality basic education. We compared the MTSF to the second draft of the APP and identified that the following indicators in the MTSF were not included in the APP

15. List the MTSF indicators related to the core function
Are there any of the MTSF indicators related to the customised indicators?
Are there any MTSF indicators related to the core function?
Comment
Percentage of schools with adequate infrastructure in line with agreed norms and standards No
Yes
The indicator is not included in the APP 2018/19 and also is not included as PPM
Policy detailing the role of Universal and Verification ANA and analysis published
The Department of Basic education is re-modelling Annual National Assessment
Items used in Universal ANA are piloted a year before being used, on learners matching target population for the assessment
Learner and teacher instrument is developed and piloted to collect background information
Annual report on statistical equivalence of Universal and Verification ANA.
District ANA report produced for every district (by DBE) using Universal ANA
Percentage of target schools supplied with improved resource packs
Policy, detailed plans & strategies developed by June 2018 & critical preparatory strategies launched
Human Resource Capacity Plan for ECD practitioners.
Human Resource Plan for ECD practitioners

16. List the MTSF indicators related to the core function
Are there any of the MTSF indicators related to the customised indicators?
Are there any MTSF indicators related to the core function?
Comment
Proportion of principals appointed based on competency assessment processes No
 Yes
The indicator is not included in the APP 2018/19 and also is not included as PPM
Proportion of principals who have signed performance agreements
Yes
Percentage of schools with full set of financial management responsibilities on the basis of assessment
Impact evaluation to assess the performance of the system against intended goals
Implementation evaluation with clear recommendations on quality outputs and improvement in relation to tracking learner movement, progress, performance and completion
Percentage of district managers whose competency has been assessed against criteria (developed below)
Complete and consistent post-provisioning policy and regulations in place & proceed with implementation and monitoring.
Proportion of NECT activities implemented (8 districts interventions on track, innovative interventions; education dialogues)

17. Inclusion of MTSF Indicators In an Annual Performance Plan for PED’s
Province
Included All
Not All Included
Eastern Cape X
Free state
Gauteng
KwaZulu Natal
Limpopo
Mpumalanga
North West
Northern Cape
Western Cape
DBE

18. Portfolio committee considerations when reviewing DBE 2018/19 APP4
Some questions the PC can consider when engaging with DBE on the 2017/18 APP review,
1. PC noted the concerns with regard to the alignment of the MTSF to the APP. Given that the present MTSF ends in 2019,
What assurance and measures can the education sector provide to the Portfolio Committee that the MTSF targets will be achieved in 2019 in the absence of performance measures in the APP?
What processes will be implemented by the DBE to improve coordination with the sector for the DBE and PEDs to also align their APPs to the MTSF?
What processes will be implemented by the DBE to ensure the enablers required such as policy and negotiations take place to meet the MTSF targets. The AGSA cited the example of Principal performance agreements, what other indicators per the MTSF require DBE intervention before implementation by PEDs.
DBE should provide the Portfolio Committee upfront with the challenges in meeting MTSF targets.
2. Explain the processes in ensuring alignment of the budget to the APP and in particular the targets set. It is of concern that PEDs set 0 targets despite the matters being significant on account of reference to the NDP and the MTSF.
3. How does the DBE ensure that the targets set by itself and the provinces are realistic i.e. over achieving by intentionally setting low targets an visa versa, under achieving by setting unreasonable and unrealistic target. This must be noted in context of the non-alignment of the budget to the APP and MTSF.

19. 4. Key committee considerations when reviewing the APP
Does the APP align with MTSF?
Is there a logical link between objectives, indicators and targets?
Does the APP include all programme performance indicators?
Quarterly targets included for indicators that need to be reported quarterly?
Are there technical indicator descriptions for ALL indicators?
Is each performance indicator well defined AND verifiable
Is there clear and reliable baseline information for targets
Is the overall alignment between the budget and APP evident and clear?
Is each performance target specific, measurable AND time bound?
Are the Strategic Plans and Annual Performance Plans aligned to the Medium-Term Strategic Framework (MTSF) outcomes and priorities?
Are indicators relevant to the institution’s mandate and the realisation of strategic goals and objectives?
Is there a logical link between objectives, indicators and targets?
Does the APP include a complete set of indicators that is relevant to the objectives that need to be achieved e.g. are all customised sector indicators relevant to the institution included in the APP?
For those programme performance indicators that need to be reported quarterly, are the quarterly targets included in the APP?
For each programme performance indicator, does the technical indicator description contain minimum required information?
Is each performance indicator well defined and verifiable, as defined in the framework?
Is there a clear identification of other data sources used to determine the baseline of the performance target in cases where the target set for the year is new and prior year's performance can therefore not be used as a baseline?
Is the overall alignment between the budget and APP evident and clear?
Does each performance target comply with the SMART criteria?

20. QUESTIONS