1. Year 3 Summary of The Memphis Striving Readers Project (MSRP) Presented by Research for Better Schools Striving Readers Annual Meeting March 22 – 23, 2010 Washington DC

2. Motivation behind MSRP

3. MSRP: Y3 Research Objectives To determine: The separate and combined effects of MCLA and READ 180 (targeted intervention) on students’ reading achievement, especially struggling readers The separate and combined effects of MCLA and READ 180 on students’ achievement in core subjects, especially struggling readers

4. Design for Estimating Impact of the Targeted Intervention READ 180 Enterprise Edition (Scholastic, Inc.) –90 minute instructional model –Used in addition to regular ELA instruction –Intended as a 2-year intervention Random assignment of struggling readers from eligible pool (bottom quartile on TCAP/Reading) 2-level HLM used to estimate impact on 7 DVs –ITBS – Total Reading, Vocabulary, Comprehension –TCAP – Math, Science, Soc. St., ELA

5. Other Variables Included in Analytic Models IV = READ 180 participation (Yes = 1; No = 0) Covariates Student LevelSchool Level Baseline score on 7 DVs (ITBS and TCAP) Percentage Female GenderPercentage African American EthnicityPercentage Special Ed FRLPercentage FRL ELLPercentage ELL Grade 7 (Y1)School Enrollment Grade 8 (Y1/Y2) Data source: ITBS and TCAP, 2006, 2007, and 2008

6. Comparability of Y3 ITT Groups, Grade 6 (N = 497)

7. Baseline Comparability on Achievement Measures

8. Comparability of Y3 ITT Groups - Grade 7 “Stayers” and “Leavers” (N=615)

9. Comparability of Y3 ITT Groups - Grade 7 “Stayers” (N = 475)

10. Comparability of Y3 ITT Groups - Grade 7 “Leavers” (N = 140)

11. Baseline Comparability on Achievement Measures

12. Comparability of Y3 ITT Groups - Grade 8 “Stayers” and “Leavers” (N= 419)

13. Comparability of Y3 ITT Groups - Grade 8 “Stayers” (N = 311)

14. Comparability of Y3 ITT Groups - Grade 8 “Leavers” (N = 108)

15. Baseline Comparability on Achievement Measures

16. Counterfactual Analyses Supplemental Reading and ELA Instruction Communication Comprehensive Reading Content Area Reading Creative Writing English Skills Failure-Free Reading Lab Language Arts Lab Reading (90 days) Reading (180 days) Tutorial English Word Bldg (180 days) Word Bldg (90 days) Word Study Skills Writing

17. Year 3 ITT Results Year 3 Sample ITBS- Read ITBS – Compr ITBS – Vocab TCAP – Read/LA TCAP - Math TCAP – Science TCAP – Soc. St. 6 th grade (N=497) 1 year R180 ns p <.05 ES =.21 Imp. = 6.9 ns 7 th grade (N=615) 2 years R180 ns 8 th grade (N=419) 2 years R180 + 1 year “business as usual” ns

18. Actual Year 3 Costs of READ 180 in Striving Readers Schools

19. READ 180 Classes Rated “Adequate” (N = 19)

20. Year 3 Barriers to Implementation N=13 Type of challengeNumber Percentag e Problems with computers969.2 Students routinely missed READ 180 due to other school-based programs/activities 430.8 Class size too large215.4 Classroom management issues17.7 Not enough class time allowed00.0 Data Source: Surveys of Read 180 teachers administered by MCS

21. Bloom’s TOT Adjustment Estimate Local Average Treatment Effect (LATE) as difference between treatment and control group means ÷ difference in compliance rates for each group LATE =

24. Impacts on Students of the Whole School Model Memphis Content Literacy Academy (MCLA)

25. Design Considerations for Estimating Y3 Impact of MCLA MCLA is designed as an intensive 2-year PD model In Y3, 2 nd cohort (former control schools) have completed half of the MCLA program (refined) Compared cohort 2 schools’ student data from Y1 with Y3 to estimate effects of being taught by teachers before (Y1) and after (Y3) one year of MCLA 2-level HLM used to estimate impact on 7 DVs –ITBS – Total Reading, Vocabulary, Comprehension –TCAP – Math, Science, Soc. St., ELA

26. Covariates Included in Analytic Models Student LevelSchool Level Baseline score on 7 DVs (ITBS and TCAP)Percentage Female GenderPercentage African American EthnicityPercentage Special Ed FRLPercentage FRL ELLPercentage ELL Grade 7 (Y1)School Enrollment Grade 8 (Y1/Y2) Data source: ITBS and TCAP, 2006, 2007, and 2008

27. Number of Students Enrolled in the Four Cohort 2 Schools by Year and Grade Comparison Group Grade 6Grade 7Grade 8 Year 1 (No MCLA)817943821 Year 3 (MCLA)875895938 Total1,6821,8381,759 Data source: MCS enrollment files, 2006 and 2008

28. Year 3 vs. Year 1 Immediate “Impact” of MCLA in Cohort 2 Schools on Spring 2009 Scores SampleITBS Tot Read ITBS Comp ITBS Vocab TCAP ELA TCAP Math TCAP Sci TCAP Soc St 6 th grade (N=1,682) ns 7 th grade (N=1,838) p <.01 ES =.55 Imp = 12.5 p <.05 ES =.36 Imp = 10.2 p <.01 ES =.54 Imp = 12.2 ns 8 th grade (N=1,759) p <.01 ES =.93 Imp = 24.3 p <.054 ES =.85 Imp = 29.0 p <.01 ES =.41 Imp = 9.8 ns p <.05 ES =.37 Imp = -5.7 Data source: ITBS and TCAP, 2006–2009

29. Schoolwide MCLA Participation and Implementation Rankings, Year 3

30. Other Analyses Underway: READ 180 IRR Study of RBS’ READ 180 Observation Protocol Analyses using Y3 counterfactual data –Time variable related to exposure to ELA-type instruction Use of PSM to reduce bias of impact estimates Y3 relationships among SRI and DVs (ITBS & TCAP) –Using SRI data from all 8 th grade students in SR schools, not just struggling readers eligible for READ 180

31. Other Analyses: MCLA Analysis of coach and teacher data to explore: –How much time coaches devote to various types of activities –Teacher perceptions of benefits from working with a coach –Whether working with a coach increases teachers’ use of SBRR –Relationships between coaching support & student achievement (completed) Use of Innovation Configuration (IC) Mapping to: –Operationally define key MCLA clusters/components –Describe/rate practices of low, medium, and high FOI teachers –Establish empirical links between key components & outcomes (ongoing)