1. Introduction to National Evaluation System
The Presidency
Department of Performance Monitoring and Evaluation
Introduction to National Evaluation System
Ian Goldman
Head: Evaluation and Research
Presentation to SAMEA
19 September2013

2. Approach – focus on accountability and use
Promoting ownership
Departments request evaluations
Evaluations implemented as partnership with DPME, co-funded, departments chair steering committee
Ensure ownership by programme manager and M&E person
Departments do management response
Cross-government evaluation technical working group supports the system, and selects the evaluations
Ensuring follow-up
Have to do improvement plans
Improvement plans monitored
Promoting accountability
All evaluation reports in the national system go to Cabinet (which approves the Plan)
Evaluations put on DPME website in evaluation repository once gone to Clusters and Cabinet
Evaluations presented to parliamentary portfolio committees

3. Key aspects of the SA national evaluation system
Strategic focus - National Evaluation Plan with 15 evaluations a year – based on importance (either by scale or because strategic or innovative)
Driver in DPME – Evaluation and Research Unit – 13 staff
Cross-government coalition - Selection by cross-government Evaluation Technical Working Group Evaluations must be made public unless security concerns
To ensure independence:
Evaluations implemented as partnership between department(s) and DPME
Steering Committee makes decisions on evaluation not department
External service providers undertake the evaluation reporting to the Steering Committee
To ensure quality:
Peer reviewers (normally 2) per evaluation
Evaluation panel, standards, guidelines, training etc
Quality assessment once completed
Role of DPME evaluation director in supporting evaluations

4. Implementation evaluation
Different types of evaluations related to questions around the outcome model
Economic Evaluation
What are the
cost-benefits?
Impact evaluation
Has the intervention had impact at outcome and impact level, and why
Implementation evaluation
- what is happening and why
If presenting the following slides on each type of evaluation, then can be brief here. Mention evaluation synthesis too.
DESIGN
Diagnostic
what is the underlying situation and root causes of the problem
Design evaluation
Does the theory of change seem strong?

5. Progress with National Evaluation System
National Evaluation Policy Framework approved November 2011
3 evaluations completed, 17 more underway, 15 more selected for 2014/15
>12 Guidelines and templates - ranging from Guideline on TORs to Guideline on Improvement Plans
Standards for evaluations and competences drafted, and standards have guided a quality assessment tool
2 courses developed, over 250 government staff trained
Evaluation panel developed with 42 organisations which simplifies procurement – however evaluation capacity is still limited – and a particular problem that few universities are bidding for evaluations
Gauteng, W Cape provinces have developed provincial evaluation plans. DPME is working with other provinces who wish to develop PEPs, starting with Free State
3 departments have developed a departmental evaluation plan (dti, DST, DRDLR)
Go to

6. Evaluation challenges emerging
Overall the system is working well but some challenges. These include:
Poor programme plans (for the government programmes which are being implemented) and so difficult to evaluate - need for minimum standards for programme plans – DPME has issued guideline on this
Poor communication channels - programme managers often not aware of the possibility of conducting evaluations on their programmes
Some senior managers wary of evaluation and don’t see it as an opportunity to improve their performance
Making sure evaluations proposed are strategic ones and that key sectors covered
Sometimes departments not budgeting for evaluations and expecting DPME to provide all the money
Departments not planning ahead – very important for impact evaluations in particular where need to plan 3+ years ahead, also affects how rollout happens
Reluctance to rollout in carefully planned way which facilitates impact evaluation. To be clear on impact must compare with/without the intervention

8. Card exercise
Groups of around 6 are seated at tables. They discuss for 20 minutes “What are the main issues affecting the use of evaluations to inform changes to government’s work, and to increase its impact?”.
Each idea is put on a separate card.
Facilitator then asks each group to read out 2 cards.
Then the next group reads two cards until all are completed