PISA FOR DEVELOPMENT International Advisory Group Meeting Closing Session May 2014 OECD Secretariat 1

PISA FOR DEVELOPMENT Conclusions

Strong representation at the meeting from Countries signed up or committed to participation in PISA for Development (Cambodia, Ecuador, Guatemala, Rwanda, Senegal, Tanzania, Zambia), Development partners (France, Germany (BMZ/GIZ), IADB, Ireland (Irish Aid), Japan(JICA),Korea, Norway, UK (DFID), World Bank), International agencies (UNESCO, UIS, UNICEF, EFA GMR, CONFEMEN and PASEC, LLECE), and Independent technicians in the field. Observers from Luxembourg and the United States PISA for Development International Advisory Group - Conclusions

Reconfirmed unanimous support for the PISA for Development project; and Acknowledged again the unique value that PISA has as an international benchmarking tool for quality and equity in schooling and for guiding policies for system improvement. Appreciated the actions taken since June 2013 Noted the indicative timeline and implementation schedule for the project and noted that this is subject to change as country schedules are established PISA for Development International Advisory Group - Conclusions

Celebrated the signing of the Participation Agreement between OECD and Ecuador and the remarks of Harvey Sanchez, Executive Director of INEVAL, who explained how participation in PISA for Development will help drive up the quality and inclusiveness of the school system in Ecuador. PISA for Development International Advisory Group - Conclusions

Adopted the ToR for the IAG and the content of the Annual Report, subject to certain amendments, including clarifying: – the respective roles and expectations of the constituent groups of the IAG – the roles between the IAG and the TAG – the overall governance of the project Noted that the IAG will have a chance to further review through written consultation Agreed to nominate co-chairs who would be elected at the next meeting of the IAG scheduled for March PISA for Development International Advisory Group - C onclusions

Endorsed the issues and challenges identified in the expert paper on enhancing the PISA cognitive instruments and Agreed on the options for adapting the cognitive instruments to fit the contexts of developing countries including in the four key aspects of review of assessment framework and items, test design, review of proficiency levels and review of scaling models. Noted that sampling and target population comparability was also an issue to seek bidders proposals on. Noted the need to ensure sufficient flexibility in order to serve national policy needs, while at the same time providing valid international benchmarking. PISA for Development International Advisory Group - Conclusions

Endorsed the issues and challenges identified in the expert paper on enhancing the PISA context questionnaires; and Agreed on the options for adapting the questionnaires to fit the contexts of developing countries including measures of early learning opportunities, language at home and school, family and community support, quality of instruction, learning time, student socioeconomic status and school resources. Noted the constraint of the length of the student questionnaire and that national optional items could be included to meet specific national needs but suggested that a common core of questionnaire items be identified to permit international comparisons Discussed the potential value and resource implications of teacher and parent questionnaires and suggested that bidders are asked to comment on the value added of including these in PISA for development. PISA for Development International Advisory Group - Conclusions

Agreed the importance of seeking approaches for including out of school 15-year-olds in the study and recognised the significant challenges involved and the possible options for addressing these. Reviewed and endorsed the first draft of the expert paper on out of school 15 year olds and noted the next steps in discussing and finalising this by October Recognised this as a distinct strand of the project and that this provided the opportunity for researching different approaches that may vary to meet country-specific needs. PISA for Development International Advisory Group - Conclusions

Agreed on the draft principles and components of Terms of Reference for the International Contractor(s) related to the cognitive instruments and contextual questionnaires with certain modifications, amplifications and additions, including: greater flexibility for bidders to respond to issues and questions and less prescription need to have a sufficient framework on which bids can be equally judged expert papers will be provided to bidders 10 PISA for Development International Advisory Group - Conclusions

bidders should be open to working with other existing assessment organisations and programmes bidders should be asked to clarify the human and financial resource implications of their proposals bidders should make use of local contractors in all aspects of the work alongside the international contractor(s) ensure there are adequate national structures in place to effectively implement PISA for Development 11 PISA for Development International Advisory Group - Conclusions

For contextual questionnaires: -need to have clear evaluation criteria with an emphasis on cross-cultural validity -bidders should make use of and reference the literature that exists on factors affecting learning outcomes in each of the regions 12 PISA for Development International Advisory Group - Conclusions

For contextual questionnaires: -bidders should propose the best respondents to target in order to get robust information on the key features -bidders should be asked to provide separately: (1) human and financial resource implications for student and school questionnaires and (2) human and financial resource implications for additional questionnaires that may be proposed 13 PISA for Development International Advisory Group - Conclusions

For cognitive instruments, Secretariat to examine the potential for including in the pool of secure items those that were rejected for main PISA Noted OECD’s plans for completing and consulting the IAG on the full ToR for the International Contractor(s) and the tendering process 14 PISA for Development International Advisory Group Conclusions

Noted the important role that PISA for Development is going to play in building capacity in the participating countries, including – for developing and conducting national and international large scale assessments, and – for using performance data to diagnose strengths and weaknesses in the education system, and for supporting school improvement efforts. PISA for Development International Advisory Group - Conclusions

Endorsed the plans for completing the capacity needs analysis and designing capacity building programmes for the participating countries Noted how the capacity building plans will be incorporated within the countries’ project implementation plans and coordinated with other strategies and support for assessment in the participating countries. Requested that the responsibilities of the International Contractor(s) for capacity building are fully reflected in the ToR. PISA for Development International Advisory Group - Conclusions

Noted and supported the plans for peer- to-peer learning, especially the high level strategic report Noted the important role that PISA for Development and PISA can play in contributing to and measuring post-2015 global educational targets that are focussed on learning. PISA for Development International Advisory Group - Conclusions

PISA FOR DEVELOPMENT Next Steps

OECD to complete the remaining planned expert papers and hold third technical workshop (OOS) OECD to prepare ToR and tendering documents for international contractor(s) and to finalise these in consultation with the IAG members OECD to sign participation agreements with all the countries participating in the project Participating countries to nominate NCs and NPMs and to prepare project implementation plans 19 Next Steps

Participating countries finalise agreements with development partners regarding contributions and support (e.g. international costs, in-country costs and activities) OECD to complete capacity needs analysis and to design capacity-building programmes for all participating countries OECD to finalise with development partners outstanding agreements for support to the project – general contributions and country-specific contributions OECD to confirm with other agencies arrangements for technical partnerships and collaboration First meeting of Technical Advisory Group after awarding of international contract 20 Next steps continued

Complete tendering process to commission international contractor (s) by OECD Project implementation: -Design, planning and coordination (including capacity assessment and capacity building planning for participating countries); -Technical development -Field trials and main data collection; -Analysis and reporting -Post-pilot governance 21 Next steps continued