Evaluation Results Missouri Reading Initiative

MRI’s Evaluation Activities: Surveys *Teacher Beliefs and Practices (pre/post) Annual Participant Questionnaire *Data Collection Test Scores Standardized Tests Classroom Assessments (DRA) MAPDemographics Special Education Information MAP Analyses *For schools beginning in 2002

MAP ANALYSES: Map analyses compare schools that have finished the MRI program with randomly chosen samples of non-MRI elementary schools Results indicate MRI schools generally outperform non-MRI schools

Notes for MAP Analyses Note: With all the following MAP Analyses charts the numbers are not as important as the comparative performance between MRI and non-MRI schools. This is because: 1. There is a lot of variation in the data from year to year and school to school 2. The calculation of the baseline changes as more data becomes available: –For the 2002 schools 1999 was the baseline –For the 2003 schools an average of 1999/2000 was the baseline –For the 2004 schools an average of 1999/2001 was the baseline

Notes for Chart 1 1. This chart compares MRI and non-MRI schools performance on the 2002 MAP Communication Arts Index 2. Baseline year is 1999; Outcome year is Each sample has 15 schools = number of schools that finished MRI Spring Total random sample = 150 (large enough number to satisfy statistical significance, high confidence levels)

Notes for Chart 2 1. This chart compares MRI and non-MRI schools performance on the 2002 MAP Reading Index 2. Baseline year is 1999; Outcome year is Each sample has 15 schools = number of schools that finished MRI Spring Total random sample = 150 (large enough number to satisfy statistical significance, high confidence levels)

Notes for Chart 3 1. This chart compares MRI and non-MRI schools performance on the 2003 MAP Communication Arts Index 2. Baseline year is an average of 1999/2000 (smoothes out variations); Outcome year is Each sample has 20 schools = number of schools that finished MRI Spring Total random sample 200 (large enough number to satisfy statistical significance, high confidence levels)

Notes for Chart 4 1. This chart compares MRI and non-MRI schools performance on the 2003 MAP Reading Index 2. Baseline year is an average of 1999/2000 (smoothes out variations); Outcome year is Each sample has 20 schools = number of schools that finished MRI Spring Total sample 200 (large enough number to satisfy statistical significance, high confidence levels)

Notes for Chart 5 1. This chart compares MRI and non-MRI schools performance on the 2004 MAP Communication Arts Index 2. Baseline year is an average of 1999/2001 (smoothes out variations); Outcome year is Each sample has 27 schools = number of schools that finished MRI Spring Total random sample 270 (large enough number to satisfy statistical significance, high confidence levels)

Notes for Chart 6 1. This chart compares MRI and non-MRI schools performance on the 2004 MAP Reading Index 2. Baseline year is an average of 1999/2001 (smoothes out variations); Outcome year is Each sample has 27 schools = number of schools that finished MRI Spring Total sample 270 (large enough number to satisfy statistical significance, high confidence levels)

Adequate Yearly Progress As mandated by federal law, Missouri schools must make yearly progress goals in MAP scores For 3 rd Grade Communication Arts those goals were defined as 19.4% of students achieving levels of Proficient or better in 2003, and 20.4% for The following Table provides a comparison between MRI schools and state-wide results.

MRIState % (60 / 74) 50.9% (1,046 / 2,053) % (50 / 50) 57.4% (1,167 / 2,053) Percentage of Schools Meeting AYP Levels 2003=19.4% 2004=20.4% Proficient and Advanced

Participant Survey Participants rate the usefulness of component utilization, practice change, "buy in", attitudes toward the program and trainer, etc. Results drive program change; e.g., Orientation

Notes for Participant Survey Slide This slide introduces the survey and its uses. The table in the next slide demonstrates how the survey is often used. In this case: survey respondents identified the problem of being “overwhelmed” 2. Program responded by redesigning orientation and other details 3. Program satisfaction improved from 02 to 04

Special Education We track the effects of MRI on Special Education in two ways: 1. Beginning with schools that started MRI in the Fall of 2002, all students with IEPs are identified and the type of IEP is described (Reading, Math, Speech, etc.) 2. Annual reports are made from schools about their IEP evaluation process: referrals, evaluations, and IEPs.

Notes for Special Education Chart This Chart is for schools that were in their 3 rd year Many schools do not have this data, or it is not easily accessed 9 of 18 in of 18 in of 23 in of 23 in of 34 in 2004 The data points to a decrease in referrals, evaluations, and assignment of IEP over the time schools participate in MRI. Data collection began from the onset of schools in with a complete report in 2005.

Developmental Reading Assessment (DRA) The following Table describes the changes in the percentages of cohorts of students who scored At or Above Grade Level according to the DRA at 15 MRI schools who have completed two years in the program. Key: =% At or Above GL S031=1 st Graders in Spring 2003 S042=2 nd Graders in Spring 2004 F02 =2 nd Graders in Fall 2002 S043=3 rd Graders in Spring 2004  = Percentage Change Key: S031=1st grade Spring 2003 S042=2nd Grade Spring 2004 S042=2nd Grade Spring 2004 F022=2nd Grade Fall 2002 S043=3rd Grade Spring 2004 

Change in DRA Grade Level SchoolS031S042  F022S043  % % % % % % % % % % % % % % % % % % % % % % % % % % % % % TOTALS % %