Building Capacity for effective government wide Monitoring and Evaluation Mr Oliver Seale M&E Learning Network Tuesday, 15 May 2007

2 Capacity building for Monitoring & Evaluation Background  Aims and objectives  The aim of the system is to contribute to improved governance and enhanced effectiveness of public sector institutions.  The system aims to collect, collate, analyse and disseminate information on the progress and impact of programmes.  Result Areas  Accurate information on progress in the implementation of public sector programmes is updated on an ongoing basis;  Information on the outcomes and impact achieved by government is periodically collected and presented;  The quality of monitoring and evaluation (M&E) practices in government and public bodies is continuously improved.

3 Capacity building for Monitoring & Evaluation Provincial needs analysis: example of feedback (A) 1.Little coherent or articulated strategy in provinces, though expenditure on expensive systems to collate M&E data. i.What would a coherent strategy need to contain? ii.What is an articulated strategy? What type of links are we looking for? iii.What systems are there? What do we mean by a system? iv. What data are there? How were they obtained? What is the quality of this data? v.What collation is taking place? How? Can a system collate data? What are the implications for training?

4 Capacity building for Monitoring & Evaluation Provincial needs analysis: example of feedback (B) 2. Monitoring programmes are just about collecting data, very little analysis and feedback given. i.What data, why and how are they being collected? 3. Alignment of plans doesn’t exist i.What planning does take place and how? ii.How is M&E incorporated into planning? iii.What do we mean by alignment why do we need it? iv.Is alignment always possible and necessary? What are the implications for training?

5 Capacity building for Monitoring & Evaluation Provincial needs analysis: example of feedback (C) 4. Planning without indicators i.What type of indicators do we mean and how should these be developed? ii.What indicators do exist and how are they measured? iii.How are they decided on? 5. Lack of or poor baseline data i.What baseline data do exist and do we evaluate it? ii.What type of baseline data are required? iii.How are these presently obtained? What are the implications for training?

6 Capacity building for Monitoring & Evaluation Conceptual f ramework for training 1.Description 2.Existing data bases 3.Data collection methods 4.Baseline data 1.What will be done (strategy) 2.Why will it be done (policy) 3.How will it be done (operations) 4.Indicators and criteria (how to measure) 5.When (timeframes) Monitoring Existing situation New project or programme Planning 1.System to be used (MIS) 2.Indicators 3.Methods 4.Baseline data 5.Inputs 6.Tracking i.Processes ii.Activities 7.Interventions and modifications 8.Outputs 9.Outcomes Evaluation 1.System to be used (EIS) 2.Indicators 3.Methods 4.Baseline data 5.Criteria 6.Assessment 7.Process 8.Impact 9.Lessons learned 10.Feedback

7 Capacity building for Monitoring & Evaluation Training principles ♦Integral part of planning, not an add-on  Process requires careful design in relation to priorities  Process requires participation of all involved, including beneficiaries (e.g. user-satisfaction surveys, participatory research methods) ♦Emphasis on quantification  Measurement of goals and objectives (efficacy)  Analysis of methods used (efficiency) ♦Emphasis on understanding and using existing data-bases and collecting data where necessary  Indicators should be identified from the outset ♦Documentation essential  Teach reporting and writing skills ♦M&E training incorporate planning, goal setting, problem identification, and problem solving

8 Capacity building for Monitoring & Evaluation Training principles for various levels Principles 1.Understanding of basic principles of M&E 2.Applying principles to a specific project 3.Applying principles to a programme 4.Applying principles to overall management in departments 5.Applying principles across departments/provinces 6.Actually performing evaluations Levels Basic for general users of information Basic for project managers Intermediate for programme managers Advanced for executive managers Advanced for CFOs and DDGs Specialist technical training for M&E staff

9 Capacity building for Monitoring & Evaluation Target audiences 1. Users i.Political heads and parliamentarians (incorporated into report-backs to portfolio committees) ii.Accounting officers (DGs) iii.Executive managers and managers in govt departments iv.Users of the service or the information outside government 2. Producers i.Programme managers ii.Project managers iii.Operations staff iv.Participants 3. M&E staff in national and provincial departments

10 Capacity building for Monitoring & Evaluation Examples of current provision InstitutionProgrammeDurationLevelContent University of Stellenbosch Diploma in Monitoring and Evaluation Methods 1 yearPost Graduate ♦ General principles & paradigms ♦ Clarificatory evaluation ♦ Process evaluation & programme monitoring ♦ Data collection methods ♦ Statistical and qualitative methods ♦ Impact assessment designs University of Cape Town Workshop on Monitoring and Evaluation 1 weekAll♦ General principles ♦ Measuring Public projects RegenesysShort course in Monitoring and Evaluation 3 daysNQF 4♦ Monitoring and evaluation concepts ♦ Results-based management ♦ Concepts of outcomes ♦ Project objectives and indicators ♦ Monitoring and evaluation system ♦ Team performance improvements ♦ Performance standards ♦ Risks and the impact thereof ♦ Success and failure factors ♦ Project evaluation reports

11 Capacity building for Monitoring & Evaluation Strategy and Plan of Action ♦Progress Report  Terms of reference developed for Task Team.  15 workshops on M&E for programme and project management (340 officials).  Initial needs analysis on provincial M&E capacity.  Consultation with key internal and external stakeholders.  Focus group sessions and sector-wide survey ♦Plan of Action  Determine current providers of M&E - HEIs, privates, NGOs etc. (April ‘07).  Undertake training needs analysis for M&E (May-June ‘07).  Develop M&E training programme (Sept ‘07)  Roll-out M&E training programme (Nov. ‘07).

12 Project Team Oliver Seale: Tel Ros Hirschowitz: Tel Marisa Labuschagne: Tel Siyabonga, Thank you, Rolivhuwa, Dankie, Nakhensa, Re a leboga