Missouri Reading Initiative Evaluation Plan: Goals, Activities, and Responsibilities

Overview Goals Goals Who Who What What When When Roles and Responsibilities Roles and Responsibilities

Goals I. Measure Outcomes II. Monitor Program Implementation (Process) III. Provide Useful Information for Program Development : Schools and MRI

Who? MRI Staff MRI Staff University Interns University Interns School Personnel School Personnel 5,000-7,000 Respondents: new schools new schools 4,500-7,500 students 4,500-7,500 students Teachers Teachers

What? Logic and Instruments Basic Program Logic: Step 1 ( Strategic Input): MRI provides instruction in Comprehensive Literacy... Step 2 (Strategic Objective): Classroom teachers change to (reinforce) Comprehensive Literacy practice resulting in... Step 3 (Long-Term Outcome): Improved levels of student achievement in communication arts.

Data Collection “Instruments” “Teacher Beliefs and Practices Survey”: Measures teacher beliefs and attitudes about learning styles, and how frequently they utilize certain practices in their classrooms. (Pre and Post)

Parent Involvement in Literacy Survey Parent Involvement in Literacy Survey Given in Fall and Spring to assess how schools approach parent involvement Measures changes in programs after MRI presents its parent involvement component Data Collection “Instruments”

Annual MRI Questionnaire: Set of questions with scaled responses given at year-end to teachers, administrators and other participants that focuses on implementation process. Teacher/Administrator Interviews: Semi-structured interviews that ask about the strengths and weaknesses of various program elements. (Exit)

Data Collection “Instruments” Standardized Test Scores: Standardized Test Scores: MAP and school-based scores used to document student achievement. Developmental Reading Assessment (DRA) scores: Developmental Reading Assessment (DRA) scores: An assessment tool that measures individual student reading development; a component of MRI instruction. Create Baselines and collect as implemented Create Baselines and collect as implemented

Data Collection “Instruments” Special Education and Title I data: Annual referrals to special education and Title I programs: 1.# Site team referrals 2.# Diagnostic evaluation referrals 3.#IEP placements 4.Create baselines and update annually

Data Collection “Instruments” Student data--- student age, gender, social and economic data (free/reduced lunch status, family type), ethnicity/race. Student data--- student age, gender, social and economic data (free/reduced lunch status, family type), ethnicity/race. Teacher data --- years teaching, length of service at school. Teacher data --- years teaching, length of service at school. School data --- size, percentage on free/reduced lunch, rural/urban. School data --- size, percentage on free/reduced lunch, rural/urban. Document review --- collection, review, and analysis of documents relating to program design, budgets, implementation, school involvement, etc. Document review --- collection, review, and analysis of documents relating to program design, budgets, implementation, school involvement, etc.

Timeline: Orientation: April-May 2006 Present evaluation plan and finalize evaluation roles and responsibilities Present evaluation plan and finalize evaluation roles and responsibilities Provide each school with a thorough orientation to the MRI program activities. Provide each school with a thorough orientation to the MRI program activities.

Timeline: Year I: ( ) September-December 2006 Conduct Pre-Teachers Beliefs & Practices Survey Conduct Pre-Teachers Beliefs & Practices Survey Collect student demographic data Collect student demographic data Collect baseline student achievement scores Collect baseline student achievement scores Collect baseline DRA scores Collect baseline DRA scores Collect baseline special ed./Title I data Collect baseline special ed./Title I data Collect school information Collect school information

Timeline: Year I: ( ) January-May 2007 Update Student Achievement Scores Update Student Achievement Scores April-May 2007 Implement end-of-year questionnaire to teachers and administrators Implement end-of-year questionnaire to teachers and administrators Update student demographic info. Update student demographic info. June-August 2007 Prepare site reports/1st year report Prepare site reports/1st year report

Timeline: Year II: ( ) September 2007-May 2008 Update standardized test scores as they occur Update standardized test scores as they occur Update DRA scores as they occur Update DRA scores as they occur Update special education/Title I data Update special education/Title I data May 2008 Conduct end-of-year questionnaire (May) Conduct end-of-year questionnaire (May) Update student demographic info. Update student demographic info. June-August 2008 Prepare site reports/2nd year report Prepare site reports/2nd year report

Timeline: Year III: ( ) September 2008-May 2009 Update standardized test scores Update standardized test scores Update DRA scores Update DRA scores Update special education/Title I data Update special education/Title I data March-May 2009 Conduct post-Teacher beliefs and Practices Survey Conduct post-Teacher beliefs and Practices Survey Conduct exit interviews Conduct exit interviews Collect final student achievement data Collect final student achievement data Update student demographic info. Update student demographic info.

Timeline: Year III: ( ) June-August 2009 Prepare final site reports Prepare final site reports Fall 2009 Final Report Final Report

Roles and Responsibilities Schools Schools Collect and transmit data; Collect and transmit data; Test Scores Special Education Interviews Surveys Surveys Provide General information Provide General informationSchoolStudent Teacher Service MRI MRI Support data collection Support data collectionStaff Set up systems Design Forms Internet –Data Entry –Analysis –Reports