On the use and misuse of computers in education: Evidence from a randomized experiment in Colombia Felipe Barrera-Osorio, World Bank Leigh L. Linden, Columbia University

Motivation 1. Several initiatives in Information and communications technologies (ICT) around the world A. Personal computer growing at a 24% annually (since the year 2000) in low-income countries and 20% in middle- income countries B. Internet users growing at 63% and 46% annually in low- and middle-income countries respectively C. The World Bank Group estimates that it has funded over US$3bn of ICT components in the past five years 2. Several countries are expanding the supply of computers in schools 3. Several countries in African, Asia and Latin America are considering the $100-laptop program 4. There is very little systematic research and hard data about how impact and use of ICT for education

The Program: Computadores para Educar (Computers for Education) 1. The program started in March 2002 by the Minister of Communications 2. Objective of refurbishing computers donated by the private sector to install them in public schools 3. The program train teachers, especially in the areas of language. 4. Since its creation, the program has received computers, refurbished that have been installed in public schools in municipalities. The benefit covers teachers and more than students.

The program (cont.) 1. The program includes a training component directly in the schools for a period of 20 months. 2. The intervention has two phases A. The initial phase lasts for 8 months: computers installation and adaptation in the schools B. The second phase: training of teachers

The program (cont.) 1. Second Phase: A. Lasts one year B. It is developed by different local universities C. Train teachers and coordinators on the relationship between technology and learning. a.Support the qualification of the children education in basic areas with the integration of the ICT in pedagogic projects b.Encouraging collaborative learning, creativity, and teachers’ and students’ self esteem when integrating ICTs to their pedagogic processes D. The theoretical and practical training is given in several seminars and visits to the schools.

Design of the evaluation 1. We implemented a lottery at the beginning of August 2006 on a set of 100 potential beneficiary schools; 50 beneficiaries and 50 non-beneficiaries schools come from a list that fulfilled the following set of criteria. A. the list was concentrated by geographic location: we chose schools in the north part of Antioquia, Caldas, Choco, Cordoba, Quindio y Risaralda. B. Sample schools with 80 or more students.

Design of the evaluation (cont.) 1. Data was collected at baseline survey conducted immediately after the randomization A. The baseline was conducted between August 14 and September 29, 2006 B. 49 control schools and 48 treatment schools 2. A follow-up survey A. May and June of 2008, same 97 schools of baseline. B. Out of the 13,598 students at baseline, 5,373 students went out of the sample because at baseline they were in grade 10 or 11, C. The final sample: out of 5,201 panel students 3,015 students were attritors (attrition rate of 37%). 3. The collection of information was done directly at the school in an unannounced visit.

Design of the evaluation (cont.) 1. Three different questioners were applied to students, teachers and principal of the school. 2. All students in grades 3 to 11 present at the school at the moment of the school visit. 3. Variables: socioeconomic characteristics (age, gender, family structure, number of siblings, work status, and allowance, among others,); school outcomes (attendance last week, number of hours of study outside the school, grades in math and language, repetition, and dropout spells in the past, among others); and attitude towards the school and the content of classes. 4. Standardized test: a short version of the national exam Saber test.

Estimation 1. Given that we have randomization, we estimated a simple model:

Estimation 1. In order to test problems of attrition:

Validation: baseline balance

Validation: baseline balance (cont.)

Validation: attrition

Results:Test scores, general

Results: test scores, gender

Results: test scores by grade

Results: other outcomes

So, why we did not find effects? 1. Hypotheses A. Computers do not impact education B. The program is having problems of implementation C. The teachers are not using the computers

Use of computers

Use of computers, continue

Use of computers: the version of the students

Use of computers: the version of the students (cont)

Conclusions 1. First, the program is increasing the number and use of computers in schools 2. The use of computers is, however, restricted to the informatics or computer class. 3. Despite the effort of the program in implementing the use of computers for pedagogic, there is no evidence of such use 4. The program is not having any impact on a large set of variables 5. Computers alone, without real changes in classroom use of computers, does not impact educational outcomes.