A MERICAN I NSTITUTES FOR R ESEARCH An Analysis of the Research and Impact of ICT in Education in Developing Country Contexts Nitika Tolani-Brown, PhD Meredith McCormac, MA Roy Zimmermann, PhD

A MERICAN I NSTITUTES FOR R ESEARCH Background Within the education sector, the importance of ICT lies in its ability to provide increased access to knowledge and services that such tools provide marginalized or traditionally underserved populations. Within the education sector, the importance of ICT lies in its ability to provide increased access to knowledge and services that such tools provide marginalized or traditionally underserved populations. There is a sense of positive impact among practitioners but no rigorous research to support “feeling” There is a sense of positive impact among practitioners but no rigorous research to support “feeling” In the absence of rigorous experimental literature, it is easy to persist in the notion that ICT is “a magic bullet that will provide the answer to long standing educational challenges” In the absence of rigorous experimental literature, it is easy to persist in the notion that ICT is “a magic bullet that will provide the answer to long standing educational challenges” This review is a first step to establishing recommendations and developing a research agenda for academicians, practitioners and policymakers to estimate impacts of ICT on educational outcomes in the developing world This review is a first step to establishing recommendations and developing a research agenda for academicians, practitioners and policymakers to estimate impacts of ICT on educational outcomes in the developing world

A MERICAN I NSTITUTES FOR R ESEARCH Scope Developing country contexts Developing country contexts Experimental, quasi-experimental and qualitative/case studies Experimental, quasi-experimental and qualitative/case studies Educational outcomes for learners, excluding impacts on other beneficiaries Educational outcomes for learners, excluding impacts on other beneficiaries Clear consensus has not emerged on the direction or the magnitude of ICTs effects on studentsClear consensus has not emerged on the direction or the magnitude of ICTs effects on students Policymakers need to know more about the combinations of factors that enhance or mitigate the effects of ICTs on students in educational settings if they are to make decisions based on evidence rather than intuition. Policymakers need to know more about the combinations of factors that enhance or mitigate the effects of ICTs on students in educational settings if they are to make decisions based on evidence rather than intuition.

A MERICAN I NSTITUTES FOR R ESEARCH Methodology Literature Review Literature Review Qualitative Interviews with Experts Qualitative Interviews with Experts Contacted 40 experts, including researchers, academicians, policymakers and practitioners in the US, Europe and developing countries (e.g. Zambia, Nepal and Afghanistan)Contacted 40 experts, including researchers, academicians, policymakers and practitioners in the US, Europe and developing countries (e.g. Zambia, Nepal and Afghanistan) 25 interviews conducted25 interviews conducted

A MERICAN I NSTITUTES FOR R ESEARCH Qualitative Literature Small pool of purely qualitative evaluations within our criteria Small pool of purely qualitative evaluations within our criteria Our report highlights three evaluations that represent a diverse set of approaches and methodological rigor within the qualitative arena Our report highlights three evaluations that represent a diverse set of approaches and methodological rigor within the qualitative arena Methodologies included in-depth structured interviews or focus group discussions with program beneficiaries at the local and policy levels, classroom and school observations, and literature reviews Methodologies included in-depth structured interviews or focus group discussions with program beneficiaries at the local and policy levels, classroom and school observations, and literature reviews Case studies alone do not provide a rigorous enough base from which to draw concrete programmatic and policy recommendations or impact estimates Case studies alone do not provide a rigorous enough base from which to draw concrete programmatic and policy recommendations or impact estimates Need for replication – a study’s findings may not be relevant after a few years Need for replication – a study’s findings may not be relevant after a few years

A MERICAN I NSTITUTES FOR R ESEARCH Experimental Evaluations Very few experimental evaluations have been conducted on the impact of ICT on student outcomes Very few experimental evaluations have been conducted on the impact of ICT on student outcomes Experimental Evaluations are important: Experimental Evaluations are important: Provide internal and external validity and generalizabilityProvide internal and external validity and generalizability RCTs can help identify differential effects across subgroups that might otherwise remain uncovered RCTs can help identify differential effects across subgroups that might otherwise remain uncovered Boys versus girls, special needs students, socio- economic status, etc Boys versus girls, special needs students, socio- economic status, etc Report highlights three key RCTs conducted in India on computer assisted learning programs Report highlights three key RCTs conducted in India on computer assisted learning programs Results suggest the importance of individualizing ICT solutions and investigating how ICT solutions interact with the learning environment and subsequently impact student outcomes Results suggest the importance of individualizing ICT solutions and investigating how ICT solutions interact with the learning environment and subsequently impact student outcomes

A MERICAN I NSTITUTES FOR R ESEARCH Challenges to Evaluation in Developing Countries Absence of standard framework for evaluation of ICT programs in educational settings Absence of standard framework for evaluation of ICT programs in educational settings Limited local capacity for evaluation Limited local capacity for evaluation Limited funding/scarce resources Limited funding/scarce resources Lack of demand and ownership of evaluation Lack of demand and ownership of evaluation

A MERICAN I NSTITUTES FOR R ESEARCH Lessons Learned & Recommendations Impacts of ICT are varied in literature Impacts of ICT are varied in literature Positive, negative and insignificant impacts on learning outcomes are reportedPositive, negative and insignificant impacts on learning outcomes are reported Research suggests type of impact often contingent upon other variables, including teacher pedagogy, infrastructure, learning environment, etc Research suggests type of impact often contingent upon other variables, including teacher pedagogy, infrastructure, learning environment, etc Additionally, design of many ICT interventions is based upon perceptions and anecdotal evidence Additionally, design of many ICT interventions is based upon perceptions and anecdotal evidence

A MERICAN I NSTITUTES FOR R ESEARCH Lessons Learned & Recommendations (cont’d) Design longitudinal studies Design longitudinal studies Look for differential rates of impact across subgroups Look for differential rates of impact across subgroups Evaluations conducted utilizing mixed methods could be highly useful Evaluations conducted utilizing mixed methods could be highly useful Develop international standards for evaluation of ICT’s impact on learning Develop international standards for evaluation of ICT’s impact on learning Explore other research areas, such as effects of learning styles, teacher training, and others Explore other research areas, such as effects of learning styles, teacher training, and others