Promoting the wellbeing of Africans through policy-relevant research on population and health 1 Impact evaluation of the East African Quality in Early Learning (EAQEL) Initiative Presenter: Moses Ngware Contributors: ERP team African Population and Health Research Center (APHRC) Supported by Hewlett/QEDC May 2010

Promoting the wellbeing of Africans through policy-relevant research on population and health 2 Content  Background  Research questions  Design & methods  Findings  Policy implications

Promoting the wellbeing of Africans through policy-relevant research on population and health 3 Background  An intervention study being implemented by AKF- Reading to Learn (RtL/scaffolding).  Role of APHRC: Impact assessment of the proposed RtL

Promoting the wellbeing of Africans through policy-relevant research on population and health  Are children in lower primary grades (1, 2 and 3) able to read and do mathematics calculations more proficiently as a result of the RtL approach?  Are there differences in proficiency for children exposed to parental involvement in the RtL Approach (core model plus) compared: to those exposed to the RtL Approach with no parental involvement (core model), to control schools?  What are the key contributing factors to these improvements in numeracy and literacy in grades 1, 2 and 3? Research Questions

Promoting the wellbeing of Africans through policy-relevant research on population and health Design - EAQEL  Randomised Control Trial (RCT)  Four districts – 2 Kenya & 2 Uganda  Treatment and control schools in each district  120 & 109 primary schools in Kenya & Uganda, respectively  229 HT, over 1000 teachers, 14,000 pupils, 7000 parents are targeted.  Intervention: i. Core module in 1 district & ii. Core module plus in the other district.  ControlTreatment District No% % Kinango Kwale Amolatar Dokolo Total

Promoting the wellbeing of Africans through policy-relevant research on population and health Sample determination, effect size & power calculations  Assumptions ( used OD software ):  Significance level = 0.05  Cell size (n) or # of pupils = 20  Effect size = 0.22 (informed by literature)  Correlation coeff. = 0.10  For a power of 80%, 100 schools needed.  For a power of 90%, 128 schools needed.

Promoting the wellbeing of Africans through policy-relevant research on population and health Data Analysis  Given that the causal effect for a single unit u cannot be observed, we aim to identify the average causal effect for the entire sample  The ATE of t (relative to c) over U (or any sub-population) is given by: ATE =E [Y 1 (u) – Y 0 (u)] = E [Y 1 (u)] – E [Y 0 (u)]

Promoting the wellbeing of Africans through policy-relevant research on population and health Data Analysis Time Treatment Outcome Treatment Group Control Group Average Treatment Effect on the Treated

Promoting the wellbeing of Africans through policy-relevant research on population and health Design Issues - Kenya EAQEL S I P YESNO YESG1G2 NOG3G4

Promoting the wellbeing of Africans through policy-relevant research on population and health Methods - instruments  Assessment tools: Single numeracy test for grades G1 – 3 Single literacy test in Kiswahili or Lang’o G1-3 Test items range from simple to difficult Curriculum & assessment experts, practitioners and researchers were involved – for content validity. All instruments were pre-tested.  Survey tools: head teacher’s questionnaire teacher questionnaire pupil questionnaire household questionnaire targeted parents/guardian

Promoting the wellbeing of Africans through policy-relevant research on population and health Sampling Process - Kenya Kenya’s districts: Kwale & Kinango Kinango TreatmentControl Non- Kensip Kensi p Non- Kensip Kensi p Kwale TreatmentControl

Promoting the wellbeing of Africans through policy-relevant research on population and health Sampling process - Uganda Uganda 2 Districts District 1: Core ModelDistrict 2: Core Model Plus Cluster of school zones Control Treatment Control

Promoting the wellbeing of Africans through policy-relevant research on population and health BS Findings- Literacy written  Literacy written (several aspects tested) - Administered orally to the pupils and were to write answers on a provided booklet (mean is out of 100%)

Promoting the wellbeing of Africans through policy-relevant research on population and health Literacy – 101 (Oral)  Oral literacy (several aspects)- Pupils were required to answer to a set of questions administered orally.

Promoting the wellbeing of Africans through policy-relevant research on population and health Literacy written – Distribution of score - Kenya  Poor performance in written literacy in Kenya- Both grades- Average % score <40%  Greater variability

Promoting the wellbeing of Africans through policy-relevant research on population and health Literacy written – Distribution of score - Uganda  Poor performance in written literacy in Kenya- Both grades- Average % score <10%

Promoting the wellbeing of Africans through policy-relevant research on population and health Literacy written – Distribution of score - Kenya  Better performance on oral literacy than written in Kenya

Promoting the wellbeing of Africans through policy-relevant research on population and health Literacy written – Distribution of score - Uganda  Better performance on oral literacy than written in Kenya

Promoting the wellbeing of Africans through policy-relevant research on population and health Oral Literacy competencies: Listening  Grade 1: High rating on Listening skills by country

Promoting the wellbeing of Africans through policy-relevant research on population and health Oral Literacy competencies: Listening  Grade 2: High rating on Listening skills by country

Promoting the wellbeing of Africans through policy-relevant research on population and health Oral Literacy competencies: speaking  Grade 1: High rating in Kenya than Uganda

Promoting the wellbeing of Africans through policy-relevant research on population and health Oral Literacy competencies: speaking  Grade 2: High rating in Kenya than Uganda

Promoting the wellbeing of Africans through policy-relevant research on population and health  Grade 1: High rating in Kenya than Uganda  Reading words scored lowly Oral Literacy competencies: Reading

Promoting the wellbeing of Africans through policy-relevant research on population and health  Grade 2: High rating in Kenya than Uganda Oral Literacy competencies: Reading

Promoting the wellbeing of Africans through policy-relevant research on population and health Class size and literacy score- Grade 1  Correlation Kenya:  Correlation Uganda:

Promoting the wellbeing of Africans through policy-relevant research on population and health  Correlation Kenya:  Correlation Uganda: Class size and literacy score- Grade 2

Promoting the wellbeing of Africans through policy-relevant research on population and health Availability of reading materials at school: - Story books

Promoting the wellbeing of Africans through policy-relevant research on population and health Household characteristics  of the household heads have primary education, with with no education  46% of the pupils are not give homework from school.  Among those given homework, 29.23% are not assisted in doing the homework- this varies by country (Kenya % and Uganda – 20.79%)  69% whose heads have no education, do not help their pupils in doing homework compared with 19% whose household heads have tertiary education

Promoting the wellbeing of Africans through policy-relevant research on population and health Literacy 101: Regression coefficients

Promoting the wellbeing of Africans through policy-relevant research on population and health Regression results [2]  Pupils who come home with homework and helped within the household score better- in Kenya  HH education: Pupils from households headed by heads with secondary and tertiary education score high- Kenya  Uganda only secondary education matters