Southeast Asia Primary Learning Metric (SEA-PLM): OVERVIEW SEAMEO Consultative Meeting on SEA-BES and SEA-PLM November 4th 2014 Jim Ackers, UNICEF Regional Education Adviser

effective reforms and interventions What is SEA-PLM? A primary education learning metrics for SEA which will provide timely, high-quality data to stakeholders and policy makers for effective reforms and interventions

Uniqueness of SEA-PLM? A regional metric focused at Primary Level (Grade 5) Uniquely SEAMEO-ASEAN – reflects Southeast Asian values Beyond reading to literacy, maths and ‘global citizenship’ Opportunity to apply innovative approaches to assessment Includes School/Student Context survey Based on national curriculum standards Building from target rather than minimum standards Embeds capacity building opportunities throughout

SEA-PLM will support SEAMEO Member Countries to: better measure and understand the status of learning achievement amongst the general population and for specific groups (e.g., boys/girls; sub-nationally; public/private sectors) use culturally appropriate metrics for formative and summative purposes that can assess 21st Century skills and critical thinking improve curriculum and its relevance to national needs and priorities (improving quality of education through system-level monitoring of learner achievement) build technical and analytical capacities of national examination and assessment staff strengthen SEAMEO-ASEAN technical collaboration on learning assessment and standards across education systems

Linking SEA-PLM to other regional assessments PISA, TIMMS and PIRLS to assess learning outcomes at lower secondary ages EGRA is increasingly used as a measure of early reading competence EGMA is used as a measure of numeracy and mathematics in the early grades East Asia Learning Assessment Study (EALAS) used national curriculum to prepare national Grade 3 & 5 metrics using SOLO Taxonomy and Rasch model East Asia Pacific - Early Childhood Development Scale. For children 3-5 years, currently being validated with 7000 children in 6 countries with HKU and ARNEC SEAMEO RECSAM’s - Southeast Asian Regional Standards for Mathematics and Science Teachers (SEARS) SEAMEO QITEP in Mathematics’ - Mathematics Regional Wide Assessment (MaRWA)

Learning from Global Learning Metric Task Force Task force recommends that education systems offer focus on mastering competencies in seven domains essential for future lives and livelihoods

Global Citizenship and the SEA-PLM 21st century skills Education for peace Civic engagement Life-skills Education

Collaboration across institutions Initial collaboration: UNICEF and SEAMEO: SEAMES and INNOTECH Further cooperation with: UNESCO Bangkok, Policy and Reform [EPR] Unit, UNESCO Bangkok UIS, Economic and Social Commission for Asia and the Pacific [ESCAP] Statistic Division [ACER], Brookings Institute, LMTF], Network on Education Quality Monitoring in the Asia-Pacific [NEQMAP] Future cooperation: As above +: Domain Technical Review Panels, MOE/National Teams, UNICEF Country Offices and SEAMEO Centres, ASEAN.

Progress: SEA-PLM (I) Short overview of steps taken in phase I: 35th SEAMEO High Officials Meeting (November 2012) Proposal on development of a Southeast Asia Primary Learning Metric (SEA-PLM) & Plan for Phase I to be completed in 2013 was presented and endorsed 47th SEAMEO Council Conference (March 2013) SEAMEO Council of Ministers took note of concept note and plan to complete Phase I in 2013 and to propose Phase II at 36th SEAMEO HOM Working Group Established (Beginning of 2013) Members: SEAMEO Secre- tariat, SEAMEO INNOTECH, UNICEF EAPRO, UNESCO (UIS and EPR Units), ACER, UN ESCAP Statistics Division Regional Experts Reference Group (RERG) Established (September 2013) RERG convened a Regional Expert Seminar on September 24-26, 2013 Desk Review Completed (December 2013) SEAMEO INNOTECH: Primary school curricula in Southeast Asian countries ACER: Experiences with primary learning metrics – and with PISA and TIMMS – in the region Progress: SEA-PLM (I) 36th SEAMEO High Officials Meeting (February 2014) Progress on phase I presented and full endorsement of phase II was given Proposed candidates for Phase 2: Brunei Darussalam, Lao PDR, Malaysia, Philippines, Thailand. Cambodia showed interest Short overview of steps taken in phase I: The Working Group was established which includes the SEAMEO Secretariat, SEAMEO INNOTECH, UNICEF EAPRO, UNESCO (UIS, Education Policy and Reform Unit), ACER, UN ESCAP Statistic Division. The Working Group was active in finalizing the Concept Note, in preparing for the RERG Seminar and in coordinating and supporting various activities related to the SEA-PLM. A Regional Experts Reference Group (RERG) was established for the proposed project, comprising of the Technical Working group institutions, and other interested regional and national level experts. The Experts’ Reference Group’s key function is to provide technical advice on key conceptual issues related to the 3 domains – literacy, numeracy and global citizenship. Through the technical assistance of the Experts Reference Group, UNICEF EAPRO and SEAMEO INNOTECH has convened a Regional Experts Seminar during 24-26 September 2013. Experts in curriculum, assessment and metric development from SEAMEO Member Countries, from specialized institutions, and from development partners participated in the Regional Experts Seminar. Completion of a desk review of primary school curricula in South-East Asian countries (conducted by SEAMEO INNOTECH) that includes: Review SEAMEO Member Countries’ National Curriculum at Primary levels – to identify shared domains and unique elements within national core curricular frameworks and compare this to the Learning Metrics Task Force’s (LMTF’s) seven Domains of Understanding. Compilation of comparable grade level curricula expectations or expected outcomes for children aged 8 and 11 years. Completion of a desk review of experiences with primary learning metrics – and with PISA and TIMMS – in the region (conducted by ACER). More specifically, the desk review includes: Mapping of what types of metrics have been introduced in SEA countries Regional experiences with primary learning metrics reviewed – including innovative approaches, issues, and lessons learned of relevance to the SEA-PLM initiative

ACER Review of Primary Learning Metrics - using a simplified framework of multilevel approaches to assessment Hybrid Assessment EGRA- Early Grade Reading Assessment EGMA_ Early Grade Mathematics Assessment EALAS- East Asia Learning Achievement Study International Assessments –are conducted in various languages and regions throughout the world and were design to explore cross-national variation in student learning, education systems and processes TIMSS- Trends in International Mathematics and Science Study PIRLS- Progress in International Reading Literacy Study Pre-PIRKS – Pre-progress in International Reading Literacy Study Regional assessments – sample students and schools in regions that share similar historical, linguistic, cultural and economic conditions to explore cross-national variation in student achievment with “like” countries PASEC - ‘Programme d'analyse des systemes educatifs de la CONFEMEN (PASEC)’ 0 Programme on the Analysis of hte Education System SACMEQ- Southern and Eastern African Consortium for Monitoring Educational Quality (SACMEQ) LLECE- Latin American Laboratory for Assessment of the Quality of Education

Approaches to assessment in SEA Countries Assessment Types Names of Tests Cambodia Hybrid, Regional and National EGRA, PASEC, LAS Lao PDR Hybrid, Regional, National EGRA, PASEC, NLS, ASLO, Indonesia EALAS, Literacy Boost, TIMSS, PIRLS, LAS Malaysia National, International PISA, TIMSS Myanmar Hybrid EALAS Philippines Hybrid, International, National EGRA, Literacy Boost, EALAS, Phil-IRI, TIMSS Singapore CRP, TIMSS, PIRLS Thailand Hybrid, National, International Nationwide Assessment, TIMSS Timor Leste EGRA, EALAS Vietnam Hybrid, National, Regional, Reading and Maths Assessments Study, EGRA, EALAS, Literacy Boost, PASEC 1) Hybrid ( combines methodologies of household surveys and a large scale assessment to addr4ess contextual needs of low and middle income countries) Cambodia (EGRA) Indonesia (Literacy Boost, EALAS) Lao PDR (EGRA) Myanmar (EALAS) Philippines (EGRA, Literacy Boost, EALAS) Timor Leste (EGRA, EALAS) Vietnam (EGRA, EALAS, Literacy Boost) 2) Regional- ‘Programme d'analyse des systemes educatifs de la CONFEMEN (PASEC)’ 0 Programme on the Analyses of hte Education System Cambodia (PASEC Lao PDR (PASEC) Vietnam (PASEC) 3) National Cambodia (Learning Assessment System) Indonesia (Learning Assessment System) Lao PDR (National Literacy Survey, Assessment of Student Learning Outcomes) Malaysia (Primary School Achievement Test) Philippine (Informal Reading Inventory) Singapore (Core Research Program) Thailand (Nationwide Assessment) Vietnam (Reading and Maths Assessment Study 4. International Indonesia - (TIMSS/PIRLS) Malaysia (TIMSS) Philippines (TIMSS) Singapore (TIMSS/PIRLS) Thailand (TIMSS)

INNOTECH’s review of Primary Education Curriculum SEAMEO INNOTECH reviewed the curriculum of 7 SEA countries using the 7 domains for Learning Outcomes under the LMTF. The review identified commonalities and unique elements within curriculum frameworks in the participating countries The contents were compared using the LMTF domains Completed review of primary school curriculum in Southeast Asia reflects the domains that SEA children are learning, specifically at grade 5 and which subject areas are the 7 domains more evident.

UNICEF’s support to SEA-PLM: Rationale EQUITY AND LEARNING: “Strengthened political commitment, accountability and national capacity to legislate, plan and budget for scaling up quality and inclusive education – learning assessment systems, early learning policies and programmes, risk informed sector plans” Unicef Strategic Plan: COMMITMENT TO LMTF: As LMTF Board Member COMPLEMENTARITY WITH OTHER ANALYTICAL WORK Learning, Education & Social Cohesion/MTBBE Out of School Children Studies in 10 countries Multi-country study on school grants with IIEP in 5 countries ELDS and ECD Scales COMMITMENT TO ENHANCING PRACTICE – enhanced LFA COMMITMENT TO REGIONAL PARTNERSHIPS – EFA to post 2015 Hybrid Assessment EGRA- Early Grade Reading Assessment EGMA_ Early Grade Mathematics Assessment EALAS- East Asia Learning Achievement Study International Assessments –are conducted in various languages and regions throughout the world and were design to explore cross-national variation in student learning, education systems and processes TIMSS- Trends in International Mathematics and Science Study PIRLS- Progress in International Reading Literacy Study Pre-PIRKS – Pre-progress in International Reading Literacy Study Regional assessments – sample students and schools in regions that share similar historical, linguistic, cultural and economic conditions to explore cross-national variation in student achievment with “like” countries PASEC - ‘Programme d'analyse des systemes educatifs de la CONFEMEN (PASEC)’ 0 Programme on the Analysis of hte Education System SACMEQ- Southern and Eastern African Consortium for Monitoring Educational Quality (SACMEQ) LLECE- Latin American Laboratory for Assessment of the Quality of Education

Two levels of engagement Country engagement Regional engagement MOE Focal Point in SC National Team In-country facilitating unit Experts for DTRP In-country training Regional seminars Field Test

Immediate next steps => NOV 2014 Ministries of Education Express interest in level of engagement from the 6 countries: country or regional Clarify available country resources to support implementation of SEA-PLM Share perspectives of how SEA-PLM can strengthen existing assessment and monitoring system(s) Share documentation, reports, copies of curricula, etc. Nominate experts for the Domain Technical Review Panels Nominate experts for the National Technical Team Final decision on MOE SEA-PLM focal point and lines of communication

Immediate next steps - => NOV 2014 In-country facilitating unit: Support for MOE’s Nomination of experts for the Domain Technical Review Panels Steering Committee: Final assessment of countries’ levels of engagement Final approval of members of Domain Technical Review Panel Members Institutional Contractor: 1st draft work plan and recommendations to SEA-PLM Steering Committee

Thank you very much