1. One Voice – One Plan Michigan Continuous School Improvement (MI-CSI)
Michigan Department of Education Office of Education Improvement and Innovation
One Voice – One Plan
Michigan Continuous School Improvement
(MI-CSI)

2. Continuous School Improvement Process
This view shows how the Do stage fits into the rest of the Continuous School Improvement Process.

3. To what extent does this represent your experience with School Improvement?
This short video clip provides schools with an opportunity to view what has frequently been the practice of school improvement or ‘school improvement on the fly.’ How might this compare to what is meant by “continuous school improvement?” How might this compare to their experience in their own buildings/districts?
An additional connector entitled “Connecting to School Improvement” (included above, next to slide) might be useful at this point.

4. Guided Conversations DO
WHY: Why is it important to strategically implement, monitor, and evaluate the School Improvement Plan?
HOW: How will we communicate the plan to all stakeholders so that they clearly understand and own their roles in implementation?
How do we build ownership of the strategy and the plan?
How will we ensure implementation with fidelity?
How is the implementation of your plan monitored and evaluated?
How is the impact of your plan monitored and evaluated?
The questions on the screen help us address the “why, how, and what” of the components of the “Do” stage. Independently jot a few notes in response to these questions, and when you are ready, share with a partner. We will then share a few responses with the whole group. After we have completed reviewing the Do stage, we will come back to these questions to see if our understanding has changed.
WHAT: What will your school look like when this plan is implemented with fidelity?
What is the expected impact on student achievement?

5. Do: Plan Requirements
ESEA and PA 25 require annual evaluation of the following:
Implementation of the plan
Student achievement results by subgroup using data from state assessments and other indicators
Modifications to plan as needed
ISDs/RESAs are required by PA25 to provide technical assistance to schools and districts to develop annual evaluations.
State and Federal law requires that we annually evaluate the implementation of our plan, as well as the impact that our plan has had on student achievement. But that is not the only reason to monitor and evaluate implementation and impact….it’s the only way to tell if what we planned to do is being done correctly and whether or not it is having the desired impact on students! In other words, it is the right thing to do!!
The specific requirements are noted on the document above next to the slide.

6. Develop School Improvement PlanDO
Implement Plan
Monitor Plan
Evaluate Plan
Plan
Develop School Improvement Plan
The second step in the Do process is to monitor the plan; monitor means to keep an eye on the implementation of the plan to ensure that it is being done with fidelity, or in the way it was meant to be implemented.

7. Reflection
What have you done to monitor the implementation and impact of your building/district plans?
Have participants reflect individually on this question.

8. Implementation is a Process!
Monitor Implementation
Evaluate Implementation
Monitor Impact
Evaluate Impact
Adult Focused
Carrie will cover slides 8 – 18.
It is critical that from the beginning of the implementation process we continuously monitor whether adults are implementing the strategy with fidelity and also continuously monitor the impact that implementation is having on students.
Student Focused

9. This slide from Dr. Doug Reeves shows data comparing what happens to student achievement when adult implementation is or is not monitored.
NOTE: Do we need more explanation about the actual data presented and the rubric scale score?
Leadership and Learning Center 2010

10. MONITOR ADULT IMPLEMENTATION AND IMPACT ON STUDENT ACHIEVEMENT
Implementation: Adult Focused
Impact: Student Focused
MONITOR
EVALUATE
MONITOR
EVALUATE
IS WHAT WE ARE DOING WORKING?
ARE WE SHOWING EVIDENCE OF STUDENT GROWTH?
What interim adjustments are suggested by implementation data? How might these adjustments affect the integrity of the results?
ARE STRATEGIES AND ACTIVITIES BEING IMPLEMENTED AS INTENDED WITH FIDELITY?
ARE WE COLLECTING & USING STUDENT AND ADULT DATA TO MODIFY & ADJUST ONGOING IMPLEMENTATION?
You should be collecting data that will help you answer the questions in the Monitor pillars based on evidence and not just on opinions. It is important to note the questions in the pillars and to talk about the evidence that you would collect to demonstrate that you are considering these factors as you implement. Do you have meeting schedules? Minutes? Ongoing student data? Walkthrough data of evidence of classroom implementation? The questions on this slide come from the MDE Draft Program Evaluation Framework.
MONITOR ADULT IMPLEMENTATION AND IMPACT ON STUDENT ACHIEVEMENT
(Formative)
IS IT WORKING?

11. Activities Connection to SPR 40/90, Interim SA/SA
Getting Ready to Implement
Implement
Monitoring Fidelity of Implementation and Impact
How will we address the targeted areas in your Process Data (SPP)?
What areas in your process data have been identified as challenge areas during your comprehensive needs assessment process?
How will we ensure readiness for implementation?
How will we ensure that staff and administrators have the knowledge and skills to implement?
POSSIBLE ACTIVITIES
Professional development around strategy
Purchase materials
Planning for implementation – Identify schedule for strategy use, personnel, mechanism for monitoring, rollout, etc.
Communication vehicles
How will we ensure successful opportunity for and implementation of the strategy?
Communication – to whom? How?
Ongoing coaching?
Observations?
Instructional technology utilized? *
Activities to support at-risk students (For Title One students)*
Parent Involvement *
*Required Components
How will we ensure the strategy is implemented with fidelity?
How will we monitor the impact on student achievement?
Walkthroughs
PLC/CASL meetings
Documentation of effective implementation
Documentation of impact
Demonstration classrooms, videos, self assessments
Gathering achievement data
You should already have identified the activities you intend to use for monitoring implementation of the strategies in your School Improvement Plan. If you find that your monitoring activities are not sufficient, you may want to go back and edit your plan.
This is the HEADER for MATRIX 1. See slide 30 for MATRIX I and directions for use.

12. Monitoring Implementation and Impact
Are you meeting on a regular basis to monitor implementation of your plan?
What does the data say when you monitored implementation?
What evidence have you collected to determine if adults are implementing with fidelity?
What evidence have you collected to determine the impact of implementation?
What adjustments are suggested by implementation and impact data? How might these adjustments affect the integrity of results?
How will you communicate progress with stakeholders?
Have participants discuss these questions in terms of past practice in their school/district, including what it would take to ensure answers to these monitoring questions that reflect high quality practices.

13. Matrix 1 is a tool used by schools for two purposes
Matrix 1 is a tool used by schools for two purposes. First, the tool helps schools determine if they have identified activities in response to deficit areas selected by schools on their School Process Rubrics: key characteristics (MDE) or indicators (AdvancED). Schools use column I to list any deficit areas selected from their SPR 40, SPR 90 or Interim Self Assessment(SA), or SA rubrics.
Second, it helps schools identify activities for a SPECIFIC STRATEGY for a GOAL under: Column II-Getting Reading to Implement, Column III-Implement and Column IV-Monitoring Fidelity of Implementation and Impact. It helps schools think comprehensively about getting ready to implement, implementing and progress monitoring. Once a school has identified its activities, it conducts a crosswalk between Column I and those activities listed in Columns II-IV to make sure they have addressed the deficit area/s.

14. Schools might consider using a document such as this to monitor the fidelity of strategy implementation.

15. This example aligns with the Strategy Implementation Guide in Slide 19
This example aligns with the Strategy Implementation Guide in Slide 19. Those doing the monitoring would use the Strategy Implementation Guide about the Gradual Release Model as the basis for collecting evidence. Those wishing to use the documents for professional learning around this idea will find them in the Resources. You might also wish to use a video such as the following to give participants practice in using the monitoring tool.

16. Schools might consider using a tool such as this to monitor the implementation of activities under each strategy.

17. Matrix 2 is designed to bring School Improvement to the classroom level.  Grade levels or Departments use Matrix 2 after completion of Matrix 1.
On the left side of this document the School Improvement team identifies the Annual Measureable Objective, the strategy being implemented, and the data tools and protocols being used to gather baseline, interim (benchmarking), and summative assessments. 
Using the activities identified in Matrix 1, the school improvement team identifies the support needed from district and building administrators for high quality implementation of the activities in the plan and completes the first two boxes in column 1.  Teachers also use this same set of activities to identify the current reality statements for those activities that apply to teachers and students, also in column 1.
On a bi-monthly basis the school improvement team and the staff use this tool to document progress  by the various stakeholders. The evidence column is provided to identify what processes or documents can serve to support statements made in Columns 2-4.

18. Checking for Understanding
What are your questions about monitoring implementation and impact?
This is a chance for individuals to ask for clarification around the topic of monitoring implementation and impact.

19. Reflection
What might you do to enhance the monitoring of the implementation and impact of your building/district plans?
Kathy (and/or Carrie)
Kathy has a reflection sheet that can be used.
Have participants individually reflect on this question and then share answers with the larger group.

20. Develop School Improvement PlanDO
Implement Plan
Monitor Plan
Evaluate Plan
Plan
Develop School Improvement Plan
The final step in the Do stage is to evaluate the plan, or to assess its effectiveness.

21. Reflection
What have you done to evaluate the implementation and impact of your building/ district plans?
Have participants individually reflect on this question.

22. Implementation is a Process!
Monitor Implementation
Evaluate Implementation
Monitor Impact
Evaluate Impact
Adult Focused
At the end of an implementation cycle, the process moves from monitoring to evaluating the implementation and impact of the plan.
Student Focused

23. EVALUATE ADULT IMPLEMENTATION AND IMPACT ON STUDENT ACHIEVEMENT
(Summative)
DID IT WORK?
Implementation: Adult Focused
Impact: Student Focused
MONITOR
EVALUATE
MONITOR
EVALUATE
IS WHAT WE ARE DOING WORKING?
ARE WE SHOWING EVIDENCE OF STUDENT GROWTH?
What interim adjustments are suggested by implementation data?
How might these adjustments affect the integrity of the results?
WAS THE PROGRAM IMPLEMENTED AS INTENDED?
DID WE IMPLEMENTTHE PLAN/STRATEGIES CORRECTLY & CONSISTENTLY?
DID WE GIVE IT ENOUGH TIME? RESOURCES?
DID OUR STRATEGIES RESULT IN INCREASED STUDENT ACHIEVEMENT?
What unintended consequences (good and bad) have occurred?
SHOULD THE STRATEGY/ACTIVITY BE CONTINUED? DISCONTINUED? MODIFIED?
ARE STRATEGIES AND ACTIVITIES BEING IMPLEMENTED AS INTENDED WITH FIDELITY?
ARE WE COLLECTING & USING STUDENT AND ADULT DATA TO MODIFY & ADJUST ONGOING IMPLEMENTATION?
The purpose of Evaluation is to answer the question “Did the strategy work?” The monitoring data you have collected should enable you to answer questions such as those in the “Evaluate” columns regarding implementation and impact.
MONITOR ADULT IMPLEMENTATION AND IMPACT ON STUDENT ACHIEVEMENT
(Formative)
IS IT WORKING?

24. Evaluating Implementation and Impact
What evidence do you have that the strategy was or was not implemented with fidelity?
What evidence do you have that implementation adhered to strategies, timelines, and responsibilities?
IMPACT
What impact has the strategy had on student achievement and what is the evidence?
What impact has the strategy had on subgroups and what is the evidence?
It is critical to understand why you experienced the impact that you did. If you were successful in achieving your measureable objective, you might want to analyze what allowed you to experience that success so you can replicate it with another strategy. If you were not successful in achieving your measureable objective, you will want to analyze the reasons for your lack of success so you can make adjustments that will increase your chances for success.
Question 1: What was the readiness for implementing the strategy?
Question 2: Did participants have the knowledge and skills to implement the plan?
Question 3: Was there opportunity for high quality implementation?
Question 4: Was the strategy implemented as intended?

25. Evaluate Plan - Conclusion
Given your evidence, what adjustments are needed to your School Improvement Plan, if any?
What do we continue to do?
What do we stop doing?
What do we need to “tweak”?
As a result of the evaluation, decisions will also need to be made regarding next steps – should the school continue the strategy as implemented, adapt it, or discontinue it altogether? It is important to keep in mind the “implementation dip” – a decrease in achievement - that traditionally accompanies initial implementation of a strategy. It is critical that decisions to discontinue the strategy not be made until efforts have been made to implement the strategy with fidelity.

26. Evaluate Plan - Digging Deeper
To what extent was this the right strategy/activity to address your need?
What is needed to maintain momentum and accelerate achievement gains?
Are the benefits of the strategy/activity sufficient to justify the resources it requires?
How might these results inform the School Improvement Plan?
Here are some additional questions for the leadership team to further evaluate the selected strategy and future actions that may be taken. These questions are taken from the draft MDE Program Evaluation Framework.

27. Evaluate Plan
The process is cyclical, and evaluation data should inform the next cycle of planning.
The ultimate goal is to have improvement strategies take hold and become so internalized that they become part of the school culture.
Highlight the statements on this slide and have participants reflect on how the practices suggested in these statements align with those in their school/district. The Program Evaluation Tool can be found on the Office of Field Services website here
NOTE: This link should be on the slide and hyperlinked…it does not work as is.

28. Checking for Understanding
What are your questions about evaluating implementation and impact?
This is the final chance for individuals to ask for clarification around the topic of evaluating implementation and impact. At this point they may realize how the processes of monitoring and evaluating implementation is very intertwined with monitoring and evaluating impact. Such a realization shows that they are beginning to understand the true complexity of the School Improvement process.

29. Reflection
What might you do to enhance the evaluation of the implementation and impact of your building/district plans?
****After this slide****Lisa and Jennifer will discuss Leadership Teams, Coaching, Communication ****No PPT slides
Have participants individually reflect on this question and then share answers with the larger group.

30. One Voice – One Plan
It is also critical that the School Improvement Team structure opportunities to celebrate successes, no matter how small.
Celebrating successes reinforces valued performance and reminds the school community that however challenging, school improvement results in improved academic performance.
Wrap up slides – Lisa & Jennifer – Slides 30 – 34
It is critical for teams to take the time to celebrate what is going well and to share these accomplishments with stakeholders.

31. One Voice – One Plan
However noble, sophisticated, or enlightened proposals for change and improvement might be, they come to nothing if teachers don’t adopt them in their own classrooms and if they don’t translate them into effective classroom practices.
This quote is meant to emphasize that the true test of School Improvement is not whether the plan is a quality written document – although that is also important - but whether the plan is implemented, monitored, and evaluated in a high quality way. School Improvement Plans that sit on the shelf and do not impact classrooms are of little value.

32. Guided Conversations DO
WHY: Why is it important to strategically implement, monitor, and evaluate the School Improvement Plan?
HOW: How will we communicate the plan to all stakeholders so that they clearly understand and own their roles in implementation?
How do we build ownership of the strategy and the plan?
How will we ensure implementation with fidelity?
How is the implementation of your plan monitored and evaluated?
How is the impact of your plan monitored and evaluated?
Have the participants return to the reflection questions and talk about them at their table. As a facilitator, you can structure this discussion in whatever way you will be able to assess understanding. It is the intent that, compared to the initial exposure to these questions, the responses are deeper and demonstrate a more comprehensive knowledge of the requirements for successful implementation of any initiative.
WHAT: What will your school look like when this plan is implemented with fidelity?
What is the expected impact on student achievement?

33. Questions/Comments? Please contact:
Renie Araoz
Diane Fleming
Diane Joslin-Gould
Or visit the MDE - School Improvement website

34. These training materials and resources were developed by the Michigan Continuous School Improvement Team. We deeply appreciate their time and support.
Renie Araoz – MDE/AdvancED Michigan
Deb Asano - Marquette-Alger RESA
Lisa Bannon - Wexford-Missaukee ISD
Ben Boerkoel - Kent ISD
Elizabeth Brophy - Calhoun ISD
Deb Dunbar - Bay-Arenac ISD
Scott Felkey - Oakland Schools
Diane Fleming - MDE
Linda Forward - MDE
Lisa Guzzardo Asaro - Macomb ISD
Carrie Haubenstricker - Tuscola ISD
Fiona Hinds - AdvancED Michigan
Diane Joslin-Gould - MDE
Scott Koziol - Michigan Center Public Schools
Kathleen Miller - Shiawassee RESD
Carolyn Rakotz - Wayne RESA
Dodie Raycraft - St Joseph County ISD
Karen Ruple – MDE
Jennifer Sell-Sabsook - Charlevoix-Emmet ISD
Kathy Sergeant - AdvancED Michigan