1. Evaluation of EFA MDA and MTR
EFA Coordinators - Presentation of findings
20 November 2010

2. Evaluation purpose
to determine what progress has been made in relation to monitoring of EFA in the countries in the region
the role of (statistical) capacity-building initiatives in enabling (or hindering) that progress
to assess the impact of the EFA MDA on policy development and policy discourse and government prioritisation and decision making processes
Findings from the evaluation is intended to help identify lessons that can be learnt where progress has been made/ has not been made in relation to monitoring and in what ways the nature and intensity of support offered can be modified/ strengthened to support these countries

3. Evaluation approach
Mixed methods approach – document review; key informant interviews; country case studies and two online surveys
Consultative and undertaken in partnership with UIS-AIMS
Validation through analytical workshops with UIS-AIMS and National EFA coordinators meetings
Four phases to the evaluation with each step informing the subsequent step:
Scoping/Inception phase - development of framework and planning for the evaluation
Data gathering phase - country level case studies which informed the development of the survey instruments
Analysis phase - results and interpretation of results tested
Reporting phase

4. EFA MDA and MTR
The EFA Mid-Decade Assessment and Mid-Term Policy Review was a major global endeavour to enable participating countries:
to construct a comprehensive picture of their progress towards their own EFA goals over the period 2000 to 2005
to identify priorities and promising strategies for overcoming obstacles and accelerating progress particularly with respect to the unreached, underserved and marginalised groups, and
to revise their national plans of action accordingly.
A process similar to the MDA was undertaken in Africa, Arab states and Latin America with varying levels of success.

5. EFA MDA/MTR in the Asia Pacific region
Participative and multidisciplinary approach
A country-led approach enabling countries to tailor MDA to national circumstances
Partnership between UNESCO and UNICEF and with other EFA partners to create a ‘movement’ around EFA MDA
Creation of regional structures to facilitate coordination
Strong emphasis on building capacity – training and technical assistance at regional, sub-regional and national levels
Strong focus on sub-national disparities, in particular the identification of unreached groups
Leadership by UIS-AIMS – mandate and credibility

6. EFA MDA vis-a-vis previous assessments
Most countries had undertaken some EFA assessment prior to MDA
MDA described as ‘path breaking’; ‘a historical step’; ‘ an eye opener for governments’; ‘empowering’
EFA MDA different from previous assessments:
deeper analysis (range of indicators)
systematic and robust framework for gathering and analysing data
clear, explicit focus on the unreached
led by the institutions within the country rather than consultants
strong support from regional and local partners
capacity building focus
externally driven versus internally driven
not sufficient to produce a national report (India)

7. Drivers for country participation
Political will and commitment from senior officials or Ministers
Opportunity to take stock and see ‘how far we have come and where we need to go’
Timing of MDA aligned with path of educational reform that a country had already embarked on – MDA impetus to sustain this direction (Nepal; Uzbekistan)
Benchmark country’s progress within the wider region (e.g. India; Malaysia; China; Indonesia)
To build internal capacity and learn from experts in the field (e.g. Samoa)
To gain insights and learn from other country’s experiences in the region
Opportunity to bring about change in attitudes and behaviours of education planners and develop a more evidence based approach to policy development

8. Part 3: Overall achievements and challenges

9. Overall - key achievements
Overwhelming support for the EFA MDA process from countries – increased relevance of EFA
Described as hugely relevant, beneficial, invaluable, important and effective process
30 countries completed MDA and published a report; some opted out, others carried out the assessment but did not officially publish a report
Many countries made strategic and/or operational policy changes as a result of the MDA findings
Thematic focus on reaching the unreached huge impact on getting countries to think about the marginalised groups
‘ It shifted the emphasis from pride on enrolment rate to concern for those not enrolled’
Efficient as it built on existing data structures
Flexible as it allowed countries to develop structure for analysis – Nepal added a 7 thematic group on language; India did thematic reports around 6 goals plus some

10. Strategic and operational policy changes
Examples of instrumental impact of EFA MDA
Instrumental impact is where an evaluation has a directly observable effect, such as policy change or introduction of new initiatives. The evaluation directly affects decision-making and influences changes in the program under review – at either strategic and operational levels
Integration of EFA National Action Plan into the National Education Development Strategy
Education Strategic Plan; National Education Plans; Medium Term national plans; introduction of new legislation like the RTE
Decision to embed inclusive education into all six goals of EFA (Uzbekistan)
Introduction of child friendly school policies (Cambodia)
Universalising the Integrated Child Development Scheme:; RTE Act(India)
Introduction of legal frameworks for child right (Uzbekistan)

11. Integration of databases or data systems (India, Thailand, Samoa)
Launch of Saakshar Bharat a scheme to promote adult education, especially of women (India)
Introduction of new ECE policies and funding (Indonesia, China)
Introduced a new monitoring mechanism and setting up of quality monitoring centre at Beijing Normal University (China)
Helped to plan and prioritise research work, i.e. Student achievement and teacher effectiveness (Samoa)
Adopting a provincial disaggregated approach to identifying and addressing inequalities in progress towards achieving EFA goals (Indonesia).

12. Key achievements (contd)
Provided a framework for data collection and monitoring
Process of doing the MDA was an instrument of change in itself – built capacity of people and systems
Role of UIS-AIMS pivotal – ensured MDA well crafted; clarity of purpose; passion and drive; technical capability; collaborative mechanisms
Strong support from UNESCO Bangkok critical to success
It was not seen as a one-off assessment as most countries we visited had initiated subsequent assessments on their own initiative – Uzbekistan, India, China, Indonesia, Cambodia
Signalled commitment from UIS to support countries to collect and analyse data that were being ‘demanded’ of countries

13. Key challenges at the national level
Political challenges surrounding disaggregation and definitions
Data collection (goals 1, 3 and 6 very challenging, gaps, inconsistencies, access to data)
Data analysis (definitions, complex indicators, variable capability)
Inter-Ministry coordination (politics; logistics; getting buy-in to EFA)
Need for greater involvement of planners and policy advisers to identify policy implications and ensure findings inform policy
Capacity:
Skills and knowledge
Limited financial and human resources
Workload (same individuals involved in most national initiatives)
Staff turnover
Sustaining engagement and commitment of those involved – most worked as volunteers; over and above their regular work; tension between political and professional commitment of members
Gaps - ‘unreached’ groups; goals 1,4 and 6 particularly challenging
Inconsistencies - administrative data and household data; national and sub-national level data
Access - held by different agencies

14. Lessons learnt for the future

15. Lessons learnt for end of 2015
Inclusion/involvement of National Statistics office early on in the process
Room for more best practice sharing and technical assistance between countries (although funding remains an issue)
Focus on ownership – not quality or accuracy
Focus on message- not final product (flexibility to draft report in local language; less focus on production of report)
Training modules for provincial and state level
Increased support for interpretation and policy analysis
Countries at different phases of development with different needs and different levels of capacity – creates need for tailored training
Where possible, EFA monitoring needs to be better integrated with country-specific education strategies

16. Key challenges for the region
Success at policy level requires engagement at Ministerial level forums – SEAMEO, Pacific Leaders Forum
Process for ‘end-of-EFA’ needs to start very soon, particularly for countries that did not complete the MDA – the time involved is significant
EFA monitoring is only one of many monitoring activities/ reports requested of UN agencies and over and above the everyday work for many Government officials – can be onerous
Need for greater alignment between development partners across global, regional and particularly at national levels

17. Spectrum of capacities and capabilities across countries in the region – how does a global body such as UIS/UNESCO respond to that diversity of need and ability and factor this into next steps?
Whose mandate is it to support countries in the interpretation of findings and policy analysis? UIS mandate is to support collection and quality of data
Need to reassess the relevance and usefulness of indicators