1. Finding Disadvantaged Schools/Students
Jordan 2012 Grade 2-3 Baseline Impact Evaluation Data
Chris Cummiskey
Amy Mulcahy-Dunn
Wednesday March 8, 11:45-1:15 Georgia 6 (South Tower) CIES 2017 Downtown Sheraton Atlanta, GA

2. Purpose: Identify disadvantaged schools at baseline.
Provide additional support to these schools (within context of the intervention).
Increase the disadvantaged students’ reading ability in treatment schools.

3. Assumptions Internal Evaluation Team
Collects relevant demographic/home/community/school/classroom information
Strong communication among Evaluation-Monitoring-Intervention teams
Treatment schools will be monitored by competent staff.
Staff with strong contextual understanding of the population of interest.
Intervention team will take the needed steps to provide the disadvantaged schools with the additional needed support.
Evaluation plans to follow same schools and/or students from disadvantaged schools [descriptive understanding].
Relevant information! This is information that the intervention team would be able to control with a certain degree.

4. Steps Define disadvantaged schools.
Identify disadvantaged schools using baseline data.
Compare Disadvantaged vs Non-Disadvantaged
Inform the monitoring team WHERE they are and WHAT they look like:
Demographic, home/community, school/classroom environment
What is out of control vs. within control of the planned intervention.
Further investigates these schools/classrooms/students/communities in more detail for stronger contextual understanding.
What do these schools/classrooms/communities look like?
Why are they struggling? And what of these factors could be controlled by the intervention group.
Intervention team adjusts their intervention according to the contextual findings.
NOTE: Be explicitly about are completely out of the control of the intervention components.

5. Step 1: Define “Disadvantaged” using Baseline
Oral Reading Fluency (Arabic)
Coef.
Std. Err. t
P>t
 n = 2,976
Grade2/3 students at treatment schools
Region [South]
North
8.9
1.0
9.4
0.0
Middle
4.7
0.9
5.0
School Gender
[All Boys]
All Girls
9.9
1.2
8.4
Mixed
7.2
8.0
Wealth index [Low]
Mid-Low
3.4
1.1
3.1
Mid-High
6.7
6.3
High
9.5
_cons
11.7
10.1
Southern Region
All Boys Schools
Lowest Wealth Index
Remember:
main purpose = improve literacy
Remember, that main point is to improve literacy
Look at wealth index

6. Step 2: Identify (Pinpoint) Disadvantaged Treatment Schools
Lowest SES
Mean class Orf<10wmp
[~34 of 150 classrooms]
Concatenate the student’s oral reading fluency and wealth index to the class level (since grade will effect ORF).

7. Step 3: Compare Disadvantaged vs. Non-Disadvantaged Schools
Non-Disadvantage
Disadvantaged
Students
n=2672
n=294
Region
South
34.81
13.61
North
36.6
26.87
Middle
28.59
59.52
School type
All boys
24.29
47.62
All girls
17.81
3.4
Mixed
57.9
48.98
Southern Region
All Boys Schools
Lowest Wealth Index
Paint a picture of the Disadvantaged Schools/Students.

8. Step 3: Compare Disadvantaged vs. Non-Disadvantaged Schools
Not in the intervention scope
Possibly in the intervention scope
Non-Disadvantage
Disadvant
Teachers
n=270
n=30
Teacher Gender
Male
21.85
14.29
Female
78.15
85.71
Non-Disadvantage
Disadvant
Students
n=2672
n=294
Time to read in Class
36.8
40.53
Someone reads at home
60.09
52.68
Ate before school
84.8
81.16
Take privated reading lessons
11.79
16.92
APPRECIATE THE LIMITATIONS OF THE Evaluation Baseline
Paint a picture of the Disadvantaged Schools/Students. BUT it is at the monitoring visit where the real understanding might be found!!!!!!!

9. Step 4: Communicate findings to Monitoring Team
Inform the monitoring team WHERE they are and WHAT they look like:
Demographic, home/community, school/classroom environment
What is out of control vs. within control of the planned intervention.

10. Step 5: Further Investigation: Stronger Contextual Understanding
What do these schools/classrooms/communities look like?
Why are under performing?
What could the intervention do to provide
relevant support?

11. Step 6: Adjust Intervention
Intervention team adjusts their intervention according to the contextual findings

12. Midline: Subsample of longitudinal data.
Is this really happening?

13. Thank you Chris Cummiskey Amy Mulcahy Dunn ccummiskey@rti.org