UNESCO Institute for Statistics Monitoring and Improving Learning in the 2030 Agenda Simple Road Map to a cross-national scale Silvia Montoya, PhD Director The Education and Developmnet Forum Conference Oxford University, UK September 17 th, 2015

UNESCO Institute for Statistics Content  The challenges of a common global metrics  Alternatives to measure SDG  The Learning Outcomes Index  Stepping stones towards

UNESCO Institute for Statistics Indicator proposals for global targets on basic education and literacy  Percentage of children who achieve minimum proficiency standards in reading and mathematics at end of: (i) primary (ii) lower secondary  Percentage of youth/adults proficient in literacy and numeracy skills  Parity indices (female/male, urban/rural, bottom/top wealth quintile] for all indicators that can be disaggregated Key criteria Relevance: reflect the most critical policy themes in the targets Alignment: construct to be measured must be valid and reliable and have the same meaning in all settings. Comparability: data are collected on a regular basis. Communicability: indicator easily understood and allows a clear narrative regarding progress towards the targets

UNESCO Institute for Statistics What does it seem to take?  The conceptual framework: the definition of a Educational Development Curve  Measurement tools that yield data on outcomes Baselines and more importantly for feedback Knowledge that repeated measurement is needed Focus on well-define outcomes and related benchmarks Awareness of the relevance of measurement for the different stakeholders  Capacity for action  Resources and sound policy making to achieve the goals

UNESCO Institute for Statistics Technical issues to solve  Many questions to answer some easier than others timeframe trade-offs (restrict some aspects in the short term while develop the first attempt at a global learning scale)?  Defining scope Domains: which learning domains to start with Would it be a developmental scale?  Target population  Geographic coverage 5

UNESCO Institute for Statistics Content  The challenges of a common global metrics  Alternatives to measure SDG  The Learning Outcomes Index  Stepping stones towards

UNESCO Institute for Statistics Monitoring: “the” options  Define benchmarks and indicators based on nationally defined standards (from national assessments)  Develop a “global” mechanism on measuring and framing indicators at one point in time i.e. snapshots at different grades / different ages? Based on current instruments  Indicators derived from an empirically developed scale 7

UNESCO Institute for Statistics Stream 1: Leveraging Existing Resources at the National Level INDICATORS: What? TIMELINE: When? How often? TECHNICAL LEAD AND DATA COLLECTION: UNESCO INSTITUTE FOR STATISTICS … 2022: Biennial new data release 2019: New data release 2016: First global database 2015/6: Initial database release (focus SSA, LAC, AP) UIS Learning Outcomes INDEX LOI Countries Regular survey on learning assessments and outcomes/ Catalogue 8 Distribution of Students by Level of Achievement Indicators on basic assessment characteristics Index of Access to Educational Information

UNESCO Institute for Statistics Stream 2: Global snapshots using cross- national learning assessments 9 INDICATORS: What? Ares/Subjects? TIMELINE: When? How often? (tb decided) TECHNICAL LEAD : UNESCO INSTITUTE FOR STATISTICS Indicator(s) in ISCED 1 Indicator(s) in ISCED 2 and/or 3 Indicator(s) for youth and adult capabilities Data collection Countires Technical approaches Cross-national initiativesAcademics and researchers Support TAGDonors When? Start discussions now Aim for global snapshots for each ISCED level by 2020 How often? Aim for once or twice between now and 2030 (for each snapshot, by ISCED- level and learning domain)

UNESCO Institute for Statistics Content  The challenges of a common global metrics  Alternatives to measure SDG  The Learning Outcomes Index  Stepping stones towards

UNESCO Institute for Statistics The Index of Learning Outcomes: methodology  The index scores range from 0 to 4. The unweighted sum of the four categories gives the Index value. Does the country have national assessments in the primary level of education? Does the country have national assessments in the secondary level of education? Did the country undertake a national assessment between 2012 and 2015? Did the country participate or is participating in an international (either global or regional) assessment? 11

UNESCO Institute for Statistics The Index of Learning Outcomes: methodology  For each question the country receives a 0 if the answer is no, or a 1 if the answer is yes.  Period of reference: the last three years (2012/2014)  Sources  The International Association for the Evaluation of Educational Achievement (IEA)  OECD  Regional Assessment  EGRA Tracker  ASER  GMR and UIS databases 12

UNESCO Institute for Statistics Which assessments are included? Learning assessments undertaken since 2000:  National assessments  International assessments  Public examinations …representative of target population

UNESCO Institute for Statistics Most countries get either the minimum or the maximum level 14

UNESCO Institute for Statistics There are not differences between income levels 15

UNESCO Institute for Statistics Higher income level is not correlated to a higher LOI score 16

UNESCO Institute for Statistics Content  The challenges of a common global metrics  Alternatives to measure SDG  The Learning Outcomes Index  Stepping stones towards

UNESCO Institute for Statistics LO Roadmap: On the same path at different speeds 2030…2025… Stream 1: Ongoing Stream 2: Global snapshots Stream 3: Towards a ULS by 2030 Indicators on basic assessment characteristics

UNESCO Institute for Statistics 19 Thank you!