1. Manufacturing, Engineering and Related Services SETA (merSETA)

2. Title: Monitoring and evaluation in a SETA environment
Name: Sebolelo Mokhobo-Nomvete
Strategy and Research Executive
LEADERS IN CLOSING THE SKILLS GAP

3. Purpose of merSETA M&E Framework
Track and measure change
Assess quality, effectiveness, efficiency and impact of skills development interventions and projects
Organisational efficiency
Integrate various existing monitoring, reporting and evaluation policies, processes and systems under a single framework
Guide the process of identifying, collecting, verifying, storing and reporting performance information (quantitative and qualitative) on a regular basis
Guide decision making and ongoing planning
LEADERS IN CLOSING THE SKILLS GAP

4. Policy and Legislative Framework
White Paper on PSET: establishment of M&E practice that informs policy and strategy decision making that is based on evidence (credible and reliable performance data); emphasis on qualitative M&E practice and impact assessment
Government-wide Monitoring and Evaluation Framework
Framework for Managing Programme Performance Information (FMPPI) - National Treasury
MTEF Guidelines – National Treasury
National Treasury Guidelines for Planning
COMMON THEME OF NT Policy Guidelines – accountability and transparency to legislators and public; clearly defined performance indicators; link between planning budgeting and reporting; collection of credible and reliable data; evidence based decision making; clarity of roles and responsibilities for managing performance information
LEADERS IN CLOSING THE SKILLS GAP

5. Purpose of merSETA M&E Framework
Track and measure change
NDP, NGP, IPAP, NSDS, SDA Mandate, labour market demand and supply
Impact (measuring contribution to employment growth and sector growth)
SSP
Impact (measuring changes in skills levels in demand in the merseta labour market)
Strategy
Outcomes (measuring 5 year goals and strategic objectives)
APP
Outputs – products, services (measuring MTEF indicators and annual targets; budget)
SLA
Targets (measuring targets agreed with DHET)
Ops Plans & Projects
Activities and inputs (measuring operational efficiencies of the organisation)
LEADERS IN CLOSING THE SKILLS GAP

6. Performance measures RESULT EG OF PERFORMANCE MEASURE
Skilled and capable workforce for merSETA industries
Employment rates; absorption rates; OJT productivity levels; levels of employment equity; job vacancy rate
Skills research and planning (incl. evaluations)
Quality of sector skills plan (breadth and depth of research studies conducted; data source credibility and reliability); quality rating of research outputs; number of research and evaluation studies
Grants management
Service standards (turn around times); alignment of grant distribution to scarce and critical skills, and firm specific demand
Learning programmes/interventions/projects implementation
Disaggregated to scarce skills/critical skills/priority skills – enrolment rates; completion rates; absorption rates; employer, employee, unemployed learner satisfaction rates wrt. merSETA quals and programmes; relevance (industry-provider partnerships); level of OJT competence
Quality assurance
Industry take up of quals; QA service efficiency standards; provider throughput rates;
Organisational capabilities
Results of internal audits and AG audits; rating the functioning of governance structures;
LEADERS IN CLOSING THE SKILLS GAP

7. Implementation considerations
M&E is the responsibility of everybody in the organisation, and it must be championed by the CEO and MANCO;
The M&E unit to design together with other units and divisions a common reporting platform that contains all the reporting processes to be followed and the concomitant information flows;
Carefully selected and negotiated strategic goals, programmes and objectives, supported by SMART indicators (well defined) and targets; to be developed by ALL (use of NT guidelines and tools to think and make decisions on what will be measured, how, who, data sources, how often data will be collected etc);
Qualitative monitoring – trends, insights, close out reports
In-quarter validation exercises
Evaluation plan up front – formative/summative; different types;
Culture change towards monitoring and reporting being a management ethos rather than a function of the M&E Unit. The M&E unit must be viewed as technical support and advisory centre/hub for QMR, MANCO, Programme Managers and Staff;
Capacity building for managers to equip them to place at the centre of their responsibilities monitoring and reporting on the implementation of their programmes;
An integrated performance management system that links individual, team and organisational performance
Limitations of DHET SLA – tool does allow for good definition and explanation of measures; truncated application of APP
LEADERS IN CLOSING THE SKILLS GAP