1. SPDG Implementation Conversations Show-Me Implementation September 14, 2010 3:00 PM – 4:00 PM, EDT

2. Review from last conversation Stages of Implementation – Exploration – Installation – Initial Implementation – Full Implementation – Innovation Two components that when combined, result in successful and sustainable outcomes – Program/Initiative (Innovation) Set of evidence-based practices Selected on: Need, fit, Resource Availability, Evidence, Readiness for Replication, Capacity to Implement – Supporting Infrastructure (Implementation) Ensuring that the interventions are implemented correctly with the “right people”, at the “right time”, in the “right amounts” (Implementation Fidelity)

4. How does your state select programs/initiatives to support at the state level? 1.Mandated by State/Federal 2.Supported/endorsed by US Department of Education 3.“Pet” project of influential individuals 4.“Tradition” (we’ve always funded that) 5.Research-based and evidence of positive student outcomes 6.Other

5. Has your state done a review (evaluation) of the programs/initiatives it supports to determine depth of implementation and impact? YES NO

6. If yes, how was it done? 1.Internal review (State-developed process) 2.External evaluator 3.Other

7. What statewide programs/initiatives do you support in your state? 1.SW-PBS 2.RTI 3.PLC 4.Literacy 5.Early Childhood 6.High Schools That Work 7.Dropout Prevention 8.Other

8. Missouri Experience Multiple programs/initiatives supported at the state level with both state and federal funds Many lacked data to show level (depth) of implementation and/or impact on student achievement Competing interests for $$$ and ??? about accountability prompted Commissioner to call for an implementation audit

9. Implementation Audit Purpose: “assess the range of implementation of educational initiatives and the relationship between the degree of implementation and changes in student achievement.” Conducted by outside evaluator (The Leadership and Learning Center) Department staff selected around 20 programs/initiatives supported at the state level State staff identified districts/buildings implementing the programs/initiatives Stratified, random sample

10. Implementation Audit Implementation was assessed based upon On-line surveys Interviews Document reviews A scoring rubric that considered – Learning context – Instructional practices – Professional development – Leadership practices Student outcomes were measured based on Missouri Assessment Program (MAP) data in language arts, math and science

11. Audit Results/Recommendations On a four-point scale (4 being the highest degree of implementation) programs/initiatives received both an average overall score and a “range of implementation” score Low implementation may not be a reflection of a poor program, but rather a reflection of “initiative fatigue” at the school and district level Depth of implementation is most clearly related to gains in student achievement (Effective program implemented deeply will show best results) Insufficient resources to effectively implement & sustain the program was the most frequently cited concern

12. Audit Results/Recommendations

13. Other implementation tools Currently use SET for SW-PBS http://pbismissouri.org/exemplar.html http://pbismissouri.org/exemplar.html In the process of developing implementation assessment tools similar to SET for Missouri Integrated Model (MIM), Response to Intervention (RtI) and Professional Learning Communities (PLC)—stay tuned Missouri Comprehensive Guidance—Internal Improvement Review document http://missouricareereducation.org/CDs/Guidanc e_Manual/AppendixD.pdf http://missouricareereducation.org/CDs/Guidanc e_Manual/AppendixD.pdf

14. Resources Missouri SW-PBS – http://pbismissouri.org/ http://pbismissouri.org/ Missouri Integrated Model (MIM) – http://mimschools.org/ http://mimschools.org/ Professional Learning Communities (PLC) – http://dese.mo.gov/divteachqual/sii/prolearning/ http://dese.mo.gov/divteachqual/sii/prolearning/ Response to Intervention (RtI) – http://dese.mo.gov/3tieredmodels/rti/ http://dese.mo.gov/3tieredmodels/rti/ Missouri Comprehensive Guidance – http://missouricareereducation.org/curr/cmd/guidanc eplacementG/mcgp.html http://missouricareereducation.org/curr/cmd/guidanc eplacementG/mcgp.html Missouri Implementation Audit—final report – http://dese.mo.gov/Appendix6.pdf http://dese.mo.gov/Appendix6.pdf

15. Questions…discussion… 15

16. Contact: Pam Williams pam.williams@dese.mo.gov Missouri Department of Elementary & Secondary Education P. O. Box 480 Jefferson City, Missouri 65102-0480 573-751-2965