1. OPEN AND DISTANCE LEARNING TEACHER TRAINING PROGRAMME-MALAWI IMPACT EVALUATION DESIGN SHORT TERM STRATEGY FOR IMPROVING STUDENTS ’ LEARNING OUTCOMES THROUGH REDUCING PUPIL QUALIFIED:TEACHER RATIOS Presentation by: The Malawi team APEIE Workshop, Accra,Ghana 14 May 2010

2. Vision statement Improve students ’ learning outcomes through the reduction of pupil:qualified teacher ratios Teacher training through Open and Distance Learning one of the means for reducing pupil:qualified ratios

3. Goal and Objective of Evaluation To compare the quality of teaching using ODL teachers vs. Conventionally trained teachers. Quality of teaching, which is probably best measured, in an objective way, by standardized tests scores. Test scores to be measured regularly using standardized tests,not only in standard/grade 8, but also in lower standards/grades, in particular grades 3- 7. Other measurable outcomes to include as enrollment and attendance.

4. Today ’ s situation 5,462 primary schools (EMIS 2008). 3.6 million students (EMIS 2008) and projected enrolment of 4.8million students by 2017. 3,500 teachers trained annually through conventional means(1+1 or commonly called IPTE) since 2007. High pupil :qualified teacher ratio(90:1), worse in some districts 135:1or above. 45,000 primary school teachers currently employed. Preferred pupil :teacher ratio of 60:1 by 2013.

5. Today ’ s situation(2) Objective of ODL teacher training project is to have a group of 16,000 teachers over 3 years (29/34 education districts). ODL teachers more attached to the community and therefore potentially more stable and motivated. ODL teachers oriented on pedagogical skills for 3 weeks before teaching..2.5 years on the job training plus distance learning. IPTE teachers trained for 1 year (face to face) then 1 year teaching practice in schools. Both IPTE and ODL teachers recruited via adverts and aptitude tests (age range-19 to 35 years).

6. Impact evaluation design Idea of randomizing ODL teachers across schools more appealing but a key practical issue with that approach. Randomization at the school level, by which schools will be allocated randomly between “Type 1 schools” and “type 2 schools”. Type 1 schools would assign ODL teachers to grades 3, 5 and 7 and NEW conventionally trained (IPTE) teachers to grades 4 and 6. Type 2 schools would assign ODL teachers to grades 4 and 6 and NEW conventionally trained (IPTE) teachers to grades 3, 5 and 7. Strict enforcement of teacher placements in schools taking part in the evaluation for the entire length of the evaluation.

7. School level randomisation Std. Type 1 schoolsType 2 schools 1234567812345678 Experienced teacher Experienced teacher or ODL Experienced or IPTE teacher Experienced teacher or ODL Experienced or IPTE teacher Experienced teacher or ODL Experienced teacher Experienced or IPTE Experienced teacher or ODL Experienced or new IPTE Experienced teacher or ODL Experienced or IPTE teacher Experienced teacher

8. Data collection One important issue with the suggested impact evaluation-the absence of a baseline. Options to alleviate this issue. 1. EMIS data, collected on a routine basis every year, will provide useful information on variables such as enrollment, teacher qualifications, infrastructure, etc. 2. Possibility to use results from the primary school leaving exam at the end of grade 8 as a proxy for overall school quality. 3. Possibility to use some of the standardized tests administered to a sample of schools in Malawi in grades 2, 3, 5 and 7 in 2008.

9. Data collection(2) To administer a standardized test EARLY (September preferably, maybe October) at the beginning of the 2010-2011 school year. To have an “almost baseline” in September-early October 2010, Yearly testing, in grades 3-7, at the end of each school year (May?). Computerized enrollment and attendance records and precise, computerized information about the allocation each teacher (ODL and new conventionally trained) to each school and each grade.

10. Length of evaluation IPTE (new conventionally trained) teachers start their job fully trained while the ODL teachers will start teaching at the same time as they will start their training. On the job distance learning for ODL teachers to take 2 ½ years. It is only after that period that it would be fair to compare ODL and newly qualified IPTE teachers. 3 batches of ODL teachers, 2010, 2011 and 2012. Implication: evaluation should continue at least until the 2015-2016 school year.

11. Length of evaluation(2) School Year 2010-11 Sep.–Oct.2010 2010-2011 May 2011 2011-2012 May 2012 2012-2013 May 2013 2013-2014 May 2014 2014-2015 May 2015 2015-2016 May 2016 Standardized tests At least 2 subjects in grades 3-7 Potentially more extended testing, grades 3-7 More extended testing, grades 3-7

12. Thank you