Development of M&E framework Capacity Building Workshop on Monitoring and Evaluating Progress in Education in the Pacific 27 – 31 October 2008 Nadi, FIJI Prepared by Nyi Nyi THAUNG, UIS (Bangkok)

Strategic sector planning Basic logical steps of the plan preparation 1. Situation analysis 2 Policy,goal and target setting 3 Formulation of priority action programs 4 Preparation of financial framework 5 Preparation of monitoring framework 6 Consolidation of draft sector plan 7 Final revision and adoption of the plan 8 Preparation of yearly operational plans 2

Strategic planning The strategic planning cycle 3

Key Questions How can we link data from MIS to information needs at the Nation Education Strategy How can we develop M&E framework What are the framework for developing indicators in education sector What does it mean by information and data?

Measurable indicator(s) Data required to produce the indicator Linkage between policy and data Policy Statement/ Goal Target(S) Benchmark Baseline Measurable indicator(s) Disaggregation Data required to produce the indicator Interactive Dynamic Data Data Data Source of data Method of collection Frequency QC

M & E Framework A document describing all these information agreed by all stakeholders Usually matrix Linkage Vertical : Goal toward data Horizontal : collaboration among all stakeholders particularly in data providers and users

What is data information indicators

Data Raw facts and number used for reference or further analyses Atomic level (basic) pieces of information that by themselves have little meaning e.g. number 23 can mean nothing or many things Several data may have to be combined to achieve required meaning / understanding State of unprocessed “A collection of facts from which conclusions may be drawn; “Statistical Data” Sometime, defining data vs information is depend on the user

Information Data that have been processed It may require several data to construct a piece of information to result to a better understanding Information is a collection of data that is understandable by itself in previous example, add another piece of data, the character string “age=” so that “age=23”, and we now have better understanding Facts or knowledge provided or learned as a result of reasoning, research, or study (data analysis)

Data vs. Information From a set of data, an analysis can be made and Information can be drawn

Indicators One way of transforming data into information is by constructing indicators. An indicator is a self-contained piece of information

Indicators

What is an indicator? An indicator: indicates a state or level acts as a gauge or meter for a specified purpose shows current situation, progress made so far and the remaining distance towards achieving the desired target Like a ‘road sign’ shows whether you are on the right road how far you have traveled, and how far you still have to go in order to reach your destination (or attain your objectives)

Why do we need to set indicators? Indicators are set: to serve as benchmark, guide to act as a “measuring rod” in comparing different population groups, localities, levels, sex, etc. to lead systematic and appropriate analyses to help defining relevant objectives and outputs Indicators are: basis for measuring progress towards objectives prerequisite for a monitoring system

What is good indicators Indicators should be smart: Specific Measurable Achievable or Attainable Result oriented, and Time-bound A GOOD INDICATOR MUST BE: Direct Objective Adequate Quantitative, where possible Disaggregated, where appropriate Practical Reliable

Education Indicators Enable management to monitor effectiveness and efficiency in the delivery of education services Enable judgments on key aspects of the functioning of the education system Useful tools to identify and measure changes in the education system over time, including the effect of planned interventions When indicators are produced on a regular basis they can reveal possible changes in response to policy actions

Identification of Indicators of Education Framework: "Education as a Production Function" The education system can be viewed as a form of “production“ which has three components: Inputs Processes Output (and Outcome) Then, basic education indicators can be identified and grouped under these categories.

Identification of Indicators of Education Framework: "Education as a Basic Social Service" If education is viewed as a “basic social service“, basic education indicators can be identified under three areas: Access Quality, and Management

Monitoring Education from Various Aspects Management, Quality and Equity Access Next Level Education System Participation Access Labour Market Drop-out

Monitoring Education from Various Aspects Access Education System Access to education Gross Intake Ratios Net Intake Ratios % of new entrants with ECCE Equity Indicators by sex Gender parity Index Next Level Participation Gross Enrolment Ratios Net Enrolment Ratios Access Output Survival rate to last grade Completion rate Performance Promotion, Repetition and drop-out rates Internal efficiency Quality Pupil-teacher ratio % of trained teachers Labour Market Drop-out

Analytical Framework Issues Analysis All Education Sub-sectors Demand Resources Access and Participation Quality and Performance Equity Output and Outcome Meeting Goals, targets Trends, changes Gaps, Differences Finding reasons

Sources of Education Data Supply-side Information Demand-side Information Schools School info Student info Teacher info Achievement EMIS Households Socio-economic Health and nutrition Relevant info for student’s participation, achievement, drop-out Awareness and value to education External efficiency (link to labour market Surveys Education providers Provision to education (finance) Management Policy Curriculum Reports Research and case studies - Findings

Basic Indicators of Education 1 Adult Literacy Rate (Aged 15+) 2 Coefficient of Internal Efficiency by Level 3 Completion Rate by Level 4 Dropout Rate by Grade 5 Gross Enrolment Ratio (GER) by Level 6 Gross Enrolment Ratio in Pre-Primary (ECCE) Programs 7 Gross Intake Rate (AIR) 8 Literacy Rate of 15-45 Year Old 9 Net Enrolment Ratio (GER) by Level 10 Net Intake Rate (NIR) 11 % of Classrooms meeting FSQL 12 % of Female Teachers by Level 13 % of Girl Enrolment by Level (Preschool, P, LS, US) 14 % of new Grade 1 intakes with Preschool experiences 15 % of Oversized Classes (41+ Pupils) by Level 16 % of Principals with School Management Training by Level 17 % of Schools Inspected during previous year by Level 18 % of Schools with Active Community Participation by Level 19 % of Schools with Adequate Sanitation Facilities by Level 20 % of Schools with both Proper Water Supply and Adequate Sanitation Facilities by Level Indicator 21 % of Schools with Computer Lab 22 % of Schools with Library 23 % of Schools with Proper Drinking Water Supply by Level 24 % of Schools with Science Lab (Lower Sec., Upper Sec.) 25 Professionally Trained Teachers by Level 26 Promotion Rate by Grade 27 Public Exp. on Education as a % of Total Government Exp. 28 Public Expenditure on Education as a % of GDP 29 Public Recurrent Expenditure on Education as a % of Total Government Recurrent Expenditure 30 Pupil-Class Ratio by Level 31 Pupil-Teacher Ratio by Level 32 Repetition Rate by Grade 33 Retention Rate by Level 34 Survival Rate to Grade 5 35 Teacher Attrition Rate by Teaching Level 36 Teachers with Required Academic Qualification by Level 37 Transition Rate from Lower Sec. to Upper Secondary Level 38 Transition Rate from Primary to Lower Secondary Level 39 Transition Rate from Upper Secondary to Tertiary Level 40 Unit Cost (public recurrent expenditure per pupil) by Level

Basic Indicators of Education Access Sr. Indicator Function 1 Gross Enrolment Ratio in Pre-Primary / Preschool (ECCE) Programs Input 2 % of new Grade 1 intakes with Preschool (ECCE) experiences 3 Gross Intake Rate (AIR) 4 Net Intake Rate (NIR) 5 Gross Enrolment Ratio (GER) by Level (primary, lower sec., upper sec.) 6 Net Enrolment Ratio (GER) by Level (primary, lower sec., upper sec.) 7 % of Girl Enrolment by Level (preschool, primary, l.sec., usec.) 8 Public Expenditure on Education as a % of GDP 9 Public Exp. on Education as a % of Total Government Exp. 10 Public Recurrent Expenditure on Education as a % of Total Government Recurrent Expenditure 11 Unit Cost (public recurrent expenditure per pupil) by Level

Basic Indicators of Education Quality Sr. Indicator Function 12 Teachers with Required Academic Qualification by Level Input 13 Professionally Trained Teachers by Level 14 % of Classrooms meeting FSQL 15 % of Schools with Library 16 % of Schools with Computer Lab 17 % of Schools with Science Lab (Lower Sec., Upper Sec.) 18 % of Schools with Proper Drinking Water Supply by Level 19 % of Schools with Adequate Sanitation Facilities by Level 20 % of Schools with both Proper Water Supply and Adequate Sanitation Facilities by Level

Basic Indicators of Education Quality Sr. Indicator Function 21 Pupil-Teacher Ratio by Level Process 22 Pupil-Class Ratio by Level 23 % of Oversized Classes (41+ Pupils) by Level 24 Promotion Rate by Grade 25 Repetition Rate by Grade 26 Dropout Rate by Grade 27 Survival Rate to Grade 5 28 Coefficient of Internal Efficiency by Level 29 Retention Rate by Level

Basic Indicators of Education Quality Sr. Indicator Function 30 Completion Rate by Level Output 31 Transition Rate from Primary to Lower Secondary Level 32 Transition Rate from Lower Secondary to Upper Secondary Level 33 Transition Rate from Upper Secondary to Tertiary Level 34 Literacy Rate of 15-45 Year Old 35 Adult Literacy Rate (Aged 15+)

Basic Indicators of Education Management Sr. Indicator Function 36 % of Female Teachers by Level Input 37 % of Principals received School Management Training by Level 38 Teacher Attrition Rate by Teaching Level Process 39 % of Schools with Active Community Participation by Level 40 % of Schools Inspected during previous year by Level

Difference use of education statistics and Indicators Planning and resource allocation Address special problems Monitoring Evaluation Accounting Management and decision making

Basic Steps in Selecting Indicators Common steps in selecting indicators for monitoring (or evaluation) purposes are: step 1 Setting clearly defined objective(s) step 2 Developing a list of possible indicators step 3 Assessing each possible indicator step 4 Selecting the “most appropriate” indicators

Some suggestions in selecting indicators Do Don’t Look for signals that have clear meaning Make use of proxies, when needed Set targets that are realistic Agree with beneficiaries and partners Look for data that is easily available; avoid major data collection Keep data sources and monitoring responsibilities in mind Lose sight of the objectives Assume that data will be available Set targets that cannot be achieved Impose or insist on any one indicator Over invest in attempts to quantify Use indicators that need expert analysis Use more indicators than necessary Don’t 1 – showing number of new schools means lose sight of measuring target of 95% children in schools