SPDG Implementation Conversations Show-Me Implementation September 14, :00 PM – 4:00 PM, EDT

Review from last conversation Stages of Implementation – Exploration – Installation – Initial Implementation – Full Implementation – Innovation Two components that when combined, result in successful and sustainable outcomes – Program/Initiative (Innovation) Set of evidence-based practices Selected on: Need, fit, Resource Availability, Evidence, Readiness for Replication, Capacity to Implement – Supporting Infrastructure (Implementation) Ensuring that the interventions are implemented correctly with the “right people”, at the “right time”, in the “right amounts” (Implementation Fidelity)

How does your state select programs/initiatives to support at the state level? 1.Mandated by State/Federal 2.Supported/endorsed by US Department of Education 3.“Pet” project of influential individuals 4.“Tradition” (we’ve always funded that) 5.Research-based and evidence of positive student outcomes 6.Other

Has your state done a review (evaluation) of the programs/initiatives it supports to determine depth of implementation and impact? YES NO

If yes, how was it done? 1.Internal review (State-developed process) 2.External evaluator 3.Other

What statewide programs/initiatives do you support in your state? 1.SW-PBS 2.RTI 3.PLC 4.Literacy 5.Early Childhood 6.High Schools That Work 7.Dropout Prevention 8.Other

Missouri Experience Multiple programs/initiatives supported at the state level with both state and federal funds Many lacked data to show level (depth) of implementation and/or impact on student achievement Competing interests for $$$ and ??? about accountability prompted Commissioner to call for an implementation audit

Implementation Audit Purpose: “assess the range of implementation of educational initiatives and the relationship between the degree of implementation and changes in student achievement.” Conducted by outside evaluator (The Leadership and Learning Center) Department staff selected around 20 programs/initiatives supported at the state level State staff identified districts/buildings implementing the programs/initiatives Stratified, random sample

Implementation Audit Implementation was assessed based upon On-line surveys Interviews Document reviews A scoring rubric that considered – Learning context – Instructional practices – Professional development – Leadership practices Student outcomes were measured based on Missouri Assessment Program (MAP) data in language arts, math and science

Audit Results/Recommendations On a four-point scale (4 being the highest degree of implementation) programs/initiatives received both an average overall score and a “range of implementation” score Low implementation may not be a reflection of a poor program, but rather a reflection of “initiative fatigue” at the school and district level Depth of implementation is most clearly related to gains in student achievement (Effective program implemented deeply will show best results) Insufficient resources to effectively implement & sustain the program was the most frequently cited concern

Audit Results/Recommendations

Other implementation tools Currently use SET for SW-PBS In the process of developing implementation assessment tools similar to SET for Missouri Integrated Model (MIM), Response to Intervention (RtI) and Professional Learning Communities (PLC)—stay tuned Missouri Comprehensive Guidance—Internal Improvement Review document e_Manual/AppendixD.pdf e_Manual/AppendixD.pdf

Resources Missouri SW-PBS – Missouri Integrated Model (MIM) – Professional Learning Communities (PLC) – Response to Intervention (RtI) – Missouri Comprehensive Guidance – eplacementG/mcgp.html eplacementG/mcgp.html Missouri Implementation Audit—final report –

Questions…discussion… 15

Contact: Pam Williams Missouri Department of Elementary & Secondary Education P. O. Box 480 Jefferson City, Missouri