Evaluation of EFA MDA and MTR EFA Coordinators - Presentation of findings 20 November 2010

Evaluation purpose to determine what progress has been made in relation to monitoring of EFA in the countries in the region the role of (statistical) capacity-building initiatives in enabling (or hindering) that progress to assess the impact of the EFA MDA on policy development and policy discourse and government prioritisation and decision making processes Findings from the evaluation is intended to help identify lessons that can be learnt where progress has been made/ has not been made in relation to monitoring and in what ways the nature and intensity of support offered can be modified/ strengthened to support these countries

Evaluation approach Mixed methods approach – document review; key informant interviews; country case studies and two online surveys Consultative and undertaken in partnership with UIS-AIMS Validation through analytical workshops with UIS-AIMS and National EFA coordinators meetings Four phases to the evaluation with each step informing the subsequent step: Scoping/Inception phase - development of framework and planning for the evaluation Data gathering phase - country level case studies which informed the development of the survey instruments Analysis phase - results and interpretation of results tested Reporting phase

EFA MDA and MTR The EFA Mid-Decade Assessment and Mid-Term Policy Review was a major global endeavour to enable participating countries: to construct a comprehensive picture of their progress towards their own EFA goals over the period 2000 to 2005 to identify priorities and promising strategies for overcoming obstacles and accelerating progress particularly with respect to the unreached, underserved and marginalised groups, and to revise their national plans of action accordingly. A process similar to the MDA was undertaken in Africa, Arab states and Latin America with varying levels of success.

EFA MDA/MTR in the Asia Pacific region Participative and multidisciplinary approach A country-led approach enabling countries to tailor MDA to national circumstances Partnership between UNESCO and UNICEF and with other EFA partners to create a ‘movement’ around EFA MDA Creation of regional structures to facilitate coordination Strong emphasis on building capacity – training and technical assistance at regional, sub-regional and national levels Strong focus on sub-national disparities, in particular the identification of unreached groups Leadership by UIS-AIMS – mandate and credibility

EFA MDA vis-a-vis previous assessments Most countries had undertaken some EFA assessment prior to MDA MDA described as ‘path breaking’; ‘a historical step’; ‘ an eye opener for governments’; ‘empowering’ EFA MDA different from previous assessments: deeper analysis (range of indicators) systematic and robust framework for gathering and analysing data clear, explicit focus on the unreached led by the institutions within the country rather than consultants strong support from regional and local partners capacity building focus externally driven versus internally driven not sufficient to produce a national report (India)

Drivers for country participation Political will and commitment from senior officials or Ministers Opportunity to take stock and see ‘how far we have come and where we need to go’ Timing of MDA aligned with path of educational reform that a country had already embarked on – MDA impetus to sustain this direction (Nepal; Uzbekistan) Benchmark country’s progress within the wider region (e.g. India; Malaysia; China; Indonesia) To build internal capacity and learn from experts in the field (e.g. Samoa) To gain insights and learn from other country’s experiences in the region Opportunity to bring about change in attitudes and behaviours of education planners and develop a more evidence based approach to policy development

Part 3: Overall achievements and challenges

Overall - key achievements Overwhelming support for the EFA MDA process from countries – increased relevance of EFA Described as hugely relevant, beneficial, invaluable, important and effective process 30 countries completed MDA and published a report; some opted out, others carried out the assessment but did not officially publish a report Many countries made strategic and/or operational policy changes as a result of the MDA findings Thematic focus on reaching the unreached huge impact on getting countries to think about the marginalised groups ‘ It shifted the emphasis from pride on enrolment rate to concern for those not enrolled’ Efficient as it built on existing data structures Flexible as it allowed countries to develop structure for analysis – Nepal added a 7 thematic group on language; India did thematic reports around 6 goals plus some

Strategic and operational policy changes Examples of instrumental impact of EFA MDA Instrumental impact is where an evaluation has a directly observable effect, such as policy change or introduction of new initiatives. The evaluation directly affects decision-making and influences changes in the program under review – at either strategic and operational levels Integration of EFA National Action Plan into the National Education Development Strategy Education Strategic Plan; National Education Plans; Medium Term national plans; introduction of new legislation like the RTE Decision to embed inclusive education into all six goals of EFA (Uzbekistan) Introduction of child friendly school policies (Cambodia) Universalising the Integrated Child Development Scheme:; RTE Act(India) Introduction of legal frameworks for child right (Uzbekistan)

Integration of databases or data systems (India, Thailand, Samoa) Launch of Saakshar Bharat a scheme to promote adult education, especially of women (India) Introduction of new ECE policies and funding (Indonesia, China) Introduced a new monitoring mechanism and setting up of quality monitoring centre at Beijing Normal University (China) Helped to plan and prioritise research work, i.e. Student achievement and teacher effectiveness (Samoa) Adopting a provincial disaggregated approach to identifying and addressing inequalities in progress towards achieving EFA goals (Indonesia).

Key achievements (contd) Provided a framework for data collection and monitoring Process of doing the MDA was an instrument of change in itself – built capacity of people and systems Role of UIS-AIMS pivotal – ensured MDA well crafted; clarity of purpose; passion and drive; technical capability; collaborative mechanisms Strong support from UNESCO Bangkok critical to success It was not seen as a one-off assessment as most countries we visited had initiated subsequent assessments on their own initiative – Uzbekistan, India, China, Indonesia, Cambodia Signalled commitment from UIS to support countries to collect and analyse data that were being ‘demanded’ of countries

Key challenges at the national level Political challenges surrounding disaggregation and definitions Data collection (goals 1, 3 and 6 very challenging, gaps, inconsistencies, access to data) Data analysis (definitions, complex indicators, variable capability) Inter-Ministry coordination (politics; logistics; getting buy-in to EFA) Need for greater involvement of planners and policy advisers to identify policy implications and ensure findings inform policy Capacity: Skills and knowledge Limited financial and human resources Workload (same individuals involved in most national initiatives) Staff turnover Sustaining engagement and commitment of those involved – most worked as volunteers; over and above their regular work; tension between political and professional commitment of members Gaps - ‘unreached’ groups; goals 1,4 and 6 particularly challenging Inconsistencies - administrative data and household data; national and sub-national level data Access - held by different agencies

Lessons learnt for the future

Lessons learnt for end of 2015 Inclusion/involvement of National Statistics office early on in the process Room for more best practice sharing and technical assistance between countries (although funding remains an issue) Focus on ownership – not quality or accuracy Focus on message- not final product (flexibility to draft report in local language; less focus on production of report) Training modules for provincial and state level Increased support for interpretation and policy analysis Countries at different phases of development with different needs and different levels of capacity – creates need for tailored training Where possible, EFA monitoring needs to be better integrated with country-specific education strategies

Key challenges for the region Success at policy level requires engagement at Ministerial level forums – SEAMEO, Pacific Leaders Forum Process for ‘end-of-EFA’ needs to start very soon, particularly for countries that did not complete the MDA – the time involved is significant EFA monitoring is only one of many monitoring activities/ reports requested of UN agencies and over and above the everyday work for many Government officials – can be onerous Need for greater alignment between development partners across global, regional and particularly at national levels

Spectrum of capacities and capabilities across countries in the region – how does a global body such as UIS/UNESCO respond to that diversity of need and ability and factor this into next steps? Whose mandate is it to support countries in the interpretation of findings and policy analysis? UIS mandate is to support collection and quality of data Need to reassess the relevance and usefulness of indicators