DHET/NSDS III RME Capacity Building Workshop RME Capacity Development Workshop/Birchwood Hotel ORT, Boksburg Antonio Hercules, Evaluations Unit/DPME 03 August 2017

Session 1 Objectives (30 mins) To submit to the workshop for consideration… To introduce a general approach to MONITORING of government initiatives, specifically NSDS III Briefly, consider a working understanding of MONITORING The monitoring ROLE of NSA specifically Approach to Monitoring Definition of Monitoring? Role and monitoring focus

Session Outcomes Policy intervention, Delivery System, OBM Participants’ agreement that MANDATE is the beginning… and understand what to monitor strategically A working understanding of monitoring, in government context Implications identified for NSA’s monitoring role, and focal monitoring elements Monitoring Concept From Concept to Monitoring System

Approach to Monitoring… of government interventions Understand the BASIC IDEA. Delivery System Outcomes (results)-based Monitoring System Policy intervention, Delivery System, OBM From Policy to Delivery From Concept to Monitoring System

Policies, Delivery Systems, OBM BASIC IDEA POLICY MANDATE: Delivery System: Outline CORE ELEMENTS, PROCESS MAP. Eg. Occupational learning sub-systems, GET/PSET linkages Onto monitoring… and evaluation role/functions SDA (1998), SDLA (1999), NQFA(2008), White Papers… etc. COMPLEX, establish BASIC IDEAS Negotiate a CONSENSUS, and REFORM policy for improved COHERENCE? Monitoring Concept

Government Proj/Prog Monitoring System Immediate Outcomes Desired change in long term KPIs Targets Monitoring focus is here Policy/prog. Goal, Objectives Main monitoring data here Main results reported here = Results Framework

Understanding Evaluations Key elements of implementation evaluations: Relevance and appropriateness: (is policy/prog relevant and aligned with related policies and frameworks? Is it appropriate for the context and especially for intended beneficiaries and stakeholders?), Effectiveness: (is the policy/prog working? Is it achieving the initial goals/objectives? Is the design appropriate? (logic, coherence, ToC, “results framework”), Current level of delivery) Efficiency: (Is the organizational design appropriate to deliver on intended goals/objectives? Adequate systems and resources? is there reasonable value-for-money, and comparability with similar policies/progs at least in the government sector? Is management and accountability mechanisms effective?)

Understanding Evaluations Key elements of implementation evaluations: Sustainability: (is the policy/prog being institutionalised in a given ministry or sector? Is there sufficient funding in the foreseeable future?) Impact: (what data do we require for an impact evaluation? What changes in action or state happened, as a direct result of our programme outputs delivered?) “In-depth and comprehensive information about the quality of service delivery”. Insights… practice improvements, and policy options.

Implementation Evaluations: sense of emphasis, importance Sustainability Efficiency Impact Effectiveness Relevance 1. Design (ToC, results framework) 2. Outputs 3. “Results” (immediate outcomes) to date 1. Org. Design 2. Manage-ment & Account-ability 3. Value-for-money? Comparison 4. Systems, Procedures 1. Institut-ionalization 2. Financial 3. Content 1. Current Data 2. Emerging Insights, Lessons 1. Relevant & aligned? 2. Appro-priate for Context and Stake-holders Independence Evidence- based Sources of Evidence: prog/dept. monitoring records, other records eg minutes, primary evaluation data Sources of Evidence: Policy documents, prog/dept. monitoring records, literature data Sources of Evidence: Policy documents, prog/dept. monitoring records, primary evaluation data Sources of Evidence: quality prog/dept. monitoring records, primary evaluation data Sources of Evidence: Policy documents, programme/dept monitoring records

Session 2: Theory of Change New linked/supporting presentation.