1. Progress Towards Reading Success: The Reading First Evaluation Prepared by: Amy Kemp, Ph.D. Research Associate and Patricia A. Muller, Ph.D. Associate Director

2. Todays Presentation Progress Towards Reading Success: The Reading First Evaluation (1) Provide overview of the three year Reading First (RF) Evaluation in Indiana (2) Present key findings from Year 1 evaluation (3) Provide details of Year 2 evaluation

3. Overview of Three Year Evaluation Progress Towards Reading Success: The Reading First Evaluation Purpose: To provide information to policy makers at IDOE and other key stakeholders to improve the implementation and impact of RF Formative: Implementation Summative: Impact

4. Formative: Implementation Progress Towards Reading Success: The Reading First Evaluation Six primary questions To what extent do RF schools implement the five essential reading components? To what extent do RF classrooms implement specific instructional strategies such as the 90 minute block, progress monitoring, etc? What types of reading instruction strategies are consistently applied in all K-3 classrooms?

5. Progress Towards Reading Success: The Reading First Evaluation What types of PD activities do the district and the school support? How well does the district and school supported PD provide for consistent, high-quality classroom instruction? What types of technical assistance are provided to the schools by the district? How effective is it?

6. Progress Towards Reading Success: The Reading First Evaluation Also…To what extent are the five goals of RF are being effectively and efficiently met? Ensuring K-3 teachers learn SBRR instruction and skills Assisting districts and schools in identifying SBRR materials, strategies, etc. Ensuring all programs and activities in K-3 classrooms meet criteria for SBRR Assisting districts and schools in selection and administration of screening, diagnostic and assessment tools Integrating initiatives and leveraging resources to avoid duplication of programs and services

7. Summative: Impact Progress Towards Reading Success: The Reading First Evaluation Primary Question/Focus: What is the impact of RF on student academic outcomes?

8. Summative: Impact Progress Towards Reading Success: The Reading First Evaluation Impact of RF on student academic outcomes Measures of student achievement (e.g. DIBELS) State standards in Reading Retention rates, special education referrals, etc.

9. Summative: Impact Progress Towards Reading Success: The Reading First Evaluation Impact of RF on student academic outcomes All K-3 students Students disaggregated based on ethnicity, ELL and special education

10. Shifting Foci Across Years Progress Towards Reading Success: The Reading First Evaluation Year 1: Assessing the level and success of implementation in Reading First schools Year 2:Assessing the degree to which RF is effectively implemented; and beginning to examine preliminary indicators of impact Year 3:Assessing the impact of RF on student outcomes

11. Year 1 Progress Towards Reading Success: The Reading First Evaluation Purpose was to assess the level and success of implementation in Reading First schools Data Collection Methods Surveys Site visits of 13 RF schools Interviews Review of extant data

12. Year 1 Findings Progress Towards Reading Success: The Reading First Evaluation At the end of the first year, the majority of schools had successfully begun to implement their RF plans Schools were using a 90-minute uninterrupted block Staff were increasingly administering assessments Teachers were using the five components of reading in their instruction Reading instruction was increasingly individualized to meet students needs

13. Year 1 Findings Progress Towards Reading Success: The Reading First Evaluation Factors critical to success included: Strong coach with content knowledge (SBRR) Strong coach with interpersonal skills (reduce resistance) Collaboration and communication among staff Climate conducive to change (support, trust, etc.) High-quality professional development (inc. confidence)

14. Year 1 Findings Progress Towards Reading Success: The Reading First Evaluation Challenges to implementation Complexity and time-intensiveness of grant requirements (trouble attending to all details) Although the majority of teachers were receptive to RF, coaches spent large amounts of time with some teachers who were having difficulty accepting the changes

15. Year 2 Evaluation Progress Towards Reading Success: The Reading First Evaluation Purpose: Assessing the degree to which RF is effectively implemented; and beginning to examine preliminary indicators of impact

16. Year 2 Evaluation Progress Towards Reading Success: The Reading First Evaluation Some Key Questions Coaches: To what extent are coaches being effectively used in schools? How are coaches spending their time? What is the role of the coach in the classroom? Core teams: To what extent are core teams being effectively used in schools? To what extent can the successful implementation of RF be attributed to core teams?

17. Year 2 Evaluation Progress Towards Reading Success: The Reading First Evaluation Changes in the classroom: To what extent are classroom instructional strategies guided by assessment results? To what extent are SBRR strategies being used in the classroom? How are the 90 minutes being used (e.g. small groups)? Flexible grouping? Interventions: To what extent are diagnostics and appropriate interventions being used in the classroom? What types of interventions are being used?

18. Year 2 Evaluation Progress Towards Reading Success: The Reading First Evaluation Professional Development: Are professional development choices being guided by classroom practice needs? Non-RF schools: To what extent has RF had an impact on non-participating schools in RF districts? To what extent are teachers at non-RF schools participating in PD, changing practices, etc.?

19. Year 2 Evaluation Progress Towards Reading Success: The Reading First Evaluation Impact Teacher level (e.g. changes in teacher knowledge/skills and classroom practices) Student Level (e.g. Are all K-3 students meeting or progressing towards their Aimlines?) What factors are associated with the effective implementation and impact of RF programs?

20. Year 2 Data Collection Progress Towards Reading Success: The Reading First Evaluation RF Consultant Interviews District Representative Interviews (All) Site visits for 13 RF programs Surveys of key stakeholders (all RF teachers, coaches, principals) Extant data (assessment data)

21. Year 2 Data Collection Progress Towards Reading Success: The Reading First Evaluation District Representative Interviews All district representatives Semi-structured phone interview Conducted late January/early February Focus on questions of program implementation and district-wide impact of the RF grant

22. Year 2 Data Collection Progress Towards Reading Success: The Reading First Evaluation Site visits for 13 RF programs Sites selected and notified by early February Two day site visits in late March/early April Observations of classrooms and relevant meetings; interviews with principals, coaches, teachers and other school staff; extant data and documents

23. Year 2 Data Collection Progress Towards Reading Success: The Reading First Evaluation Surveys of key stakeholders Includes all RF schools Teachers, coaches and principals Late April/Early May Primarily on-line surveys (with option for pencil and paper version)

24. Year 2 Data Collection Progress Towards Reading Success: The Reading First Evaluation Extant Data DIBELS and Assessment data (including progress monitoring, and diagnostic) from Wireless Generation

25. Progress Towards Reading Success: The Reading First Evaluation Prepared by: Amy Kemp, Ph.D. Research Associate and Patricia A. Muller, Ph.D. Associate Director