1. Vision: Every child in every district receives the instruction that they need and deserve…every day. Data Day 2016 Tier 2/3: Reviewing your System of Interventions

2. “Intervention failure should be an exceedingly rare event.” -VanDerHeyden & Tilly, 2010 How effective are you interventions? Roughly what percentage of students make adequate progress? Intervention group, Grade level, School, or District Record the percentage on your handout Why review our system of interventions?

3. Is our system of interventions working? 3 INTERVENTIONS (TIER 2/3) Groups of Students Individual Students

4. In order to do this, what should you have in place?

5. Interventions! –Lists of students in the different groups Data! –Graphed Progress Monitoring data –Screening data Decision rules In order to do this, what should you have in place?

6. It depends… District level School level Grade level Who does this work?

7. 1.Summary of Effectiveness Reports 2.Comparing Growth Rates 3.Examining Intervention Groups (Decision Rules) Evaluating your Interventions More Precise Less Precise More Time Consuming Less Time Consumin g

8. 1.Summary of Effectiveness Reports

9. Grade-wide Screening Data FallWinter 60%70% 20% 18% 12%

10. Grade-wide Screening Data Fall 60% 20% Receiving Interventions

11. Grade-wide Screening Data FallWinter 60%70% 20% 18% 12%

12. DIBELS Next (UO/CTL) Summary of Effectiveness 86 kids 20.8% 32 kids 7.7% 296 kids 71.5%

13. DIBELS Next (UO/CTL) Summary of Effectiveness 86 kids 64 kids (74.4%) 13 kids (15.1%) 9 kids (10.5%)

14. DIBELS Next (DIBELS NET) Effectiveness of Instructional Levels of Support 68 kids (84%) 12 kids (15%) 1 kid (1%) 81 kids

15. There currently is no Summary of Effectiveness Report built into easyCBM. Summary of Effectiveness Worksheet (Excel) posted on the ORTIi website.Summary of Effectiveness Worksheet easyCBM

16. Downloadable here 16 Update this link

17. easyCBM Benchmark Risk Analysis Reading Grade Level Sort by Change and Fall

18. 5 2 1

19. How effective is our Tier 2 and Tier 3 Level of Support?

20. ProsCons Reports already created for some data systems (DIBELS) Reports need to be created for some data systems (easyCBM) Allows you to evaluate levels of support quickly Can only be done at Winter and Spring benchmarking times Requires few prerequisites: Screening data system Summary of effectiveness Report Does not account for students who made progress but didn’t increase a level of support Pros and Cons: Summary of Effectiveness Reports

21. Activity: Summary of Effectiveness Choose one (or if you have time you can do more ): –easyCBM –DIBELS Next –Your own data, if available Use the cups to indicate your progress: Practice Time! GreenYellowRed DoneStill WorkingNeed Help

22. On a scale of 1 to 5 how well do you understand the Summary of Effectiveness Report method? Do you have the prerequisites in place? What pieces would you need to build? Who would do this work? With a neighbor 12345 I’m completely lost I have a vague understanding I know it but need some practice I know it and could confidently do it I could teach others how to do it

23. 2.Comparing Growth Rates (Rates of Improvement or ROI)

24. Reading Trajectories of Low and Middle Readers Words Per Minute © 2005, Dynamic Measurement Group

25. Based on easyCBM Percentiles (Passage Reading Fluency) 44 WPM less 55 WPM less

26. …your district/school (by grade level) …your Tier 2 intervention groups (by grade level) …your Tier 3 intervention groups (by grade level) …your subgroups Calculate average (grade level) weekly rates of improvement for…

27. How do they compare? District/School Average Oral Reading Fluency (Words Correct Per Minute) Students in Interventions Increase of 2 WCPM every week Increase of 1 WCPM per week Increase of 3 WCPM per week

28. 1.Determine average end-of-year and average beginning-of-year level of performance. 2.Calculate the difference between those two numbers. This gives you your Average yearly student growth. 3.Determine # of instructional weeks between those two data points. 4.Divide Average yearly student growth by the # of instructional weeks. Calculating your Average Weekly Growth Rates

29. Calculating your Average Weekly Growth Rates (easyCBM) Benchmark Benchmark Scores Last year Grade Level Reading Compare PRF

30. Calculating your Average Weekly Growth Rates (easyCBM)

31. CTL / UO Website (Summary Reports): DMG / DIBELS.net (School or District Overview): Calculating your Average Weekly Growth Rates (DIBELS Next) 73.795.1 71.3 101.6

32. 1.Determine average end-of-year and average beginning of year level of performance. Calculating your Average Weekly Growth Rates

34. 1.Determine average end-of-year and average beginning of year level of performance. 2.Calculate the difference between those two numbers. This gives you your Average yearly student growth. Calculating your Average Weekly Growth Rates

35. 96 – 57 = 39 Average yearly student growth

36. 1.Determine average end-of-year and average beginning of year level of performance. 2.Calculate the difference between those two numbers. This gives you your Average yearly student growth. 3.Determine # of instructional weeks between those two data points. 4.Divide Average yearly student growth by the # of instructional weeks. Calculating your Average Weekly Growth Rates

37. Average Yearly Student Growth Calculating your Average Weekly Growth Rates 39 Words per minute = # of Instructional Weeks 32 Weeks = Average Weekly Student Growth 39 / 32 = 1.2 WPM per week =

38. Students in interventions should be making MORE progress than typical students. Students in interventions are receiving MORE support than typical students. Compare your growth rates

39. Comparing Rates of Improvement Sep OctNov Dec JanFeb Mar Apr May 68 3 rd Grade DIBELS ORF 100 104 – 68 = 36 WCPM 36 WCPM / 36 Weeks = 1 WCPM per week 104 40 68 All 3 rd Graders in District (last year) All 3 rd Graders in an intervention Group 68 – 40 = 28 WCPM 28 WCPM / 20 Weeks = 1.4 WCPM per week

40. Compare your growth rates TIER 1 (DISTRICT/SCHOOL) TIER 2 INTERVENTIONSTIER 3 INTERVENTIONS All studentsAll students at Tier 2All students at Tier 3 1.2 WPM per week Phonics for Reading Groups Reading Mastery Groups Read Naturally Groups Corrective Reading Groups 1.6 WPM per week.9 WPM per week 2 WPM per week 1.4 WPM per week.5 WPM per week Caution: Only calculate the average rates of individual intervention programs if you have a more than one group 3 rd Grade

41. Aaron Eli Shannon Marie District Average 1.5 Phonics for Reading 1.75 4.5 1 1.5 Is this an effective intervention? Average weekly growth (WPM per week)

42. Pros and Cons: Comparing Growth Rates ProsCons Provides a picture of typical student growth in your district as a norm Requires some calculations Shouldn’t be done with single intervention groups Allows you to make multiple quick comparisons Requires several prerequisites: Access to a full year of screening data for district/school growth rate Formula for calculating growth rates Better for larger districts with more data

43. Activity: Comparing Growth Rates Choose one (or if you have time you can do more ): –easyCBM –DIBELS Next –Your own data, if available Use the cups to indicate your progress: Practice Time! GreenYellowRed DoneStill WorkingNeed Help

44. On a scale of 1 to 5 how well do you understand the Comparing Growth Rates method? Do you have the prerequisites in place? What pieces would you need to build? Who would do this work? With a neighbor 12345 I’m completely lost I have a vague understanding I know it but need some practice I know it and could confidently do it I could teach others how to do it

45. 3.Examining Intervention Groups (Decision Rules)

46. Reading Mastery ERI Phonics for Reading Read Naturally Phonics for Reading PALS Claire Miguel Franklin Estelle Gloria Jimmy

47. Example Modify interventions when: Progress monitoring indicates three or more data points below the aimline after 20+ instructional sessions If data is highly variable, maintain the current intervention for another month to establish a trendline –Progress is monitored weekly or biweekly. Based on our Decision Rules, which students are making adequate progress?

48. When students have “X” number of consecutive points below the aimline we must stop and think Stop and think 12 3 4

49. 50 th %ile

50. Claire Miguel Franklin Estelle Gloria Jimmy

51. Phonics for Reading Read Naturally Phonics for Reading PALS Claire Miguel Franklin Estelle Gloria Jimmy Jackson Mary Alex Sergio Grady Louise Nathan Max Cody Erin Maria Alice Ashlyn Jolene Laura Jenny Marty Saul 1 out of 6 2 out of 6 3 out of 12 6 out of 7 4 out of 5

52. Reading Mastery ERI Penny Wanda Christian Paul Amy Jose Marla Carter Kodi Eddie Francis Miles How effective are our intensive interventions?

53. Intervention Effectiveness Tool Phonics for Reading Group 1 1 16% Group 2 6 2 33% 6

54. Intervention Effectiveness Tool http://miblsi.cenmi.org/MiBLSiModel/Evaluation/Measure s/Tier23TrackingForm.aspx

55. Pros and Cons: Evaluating Intervention Groups (Decision Rules) ProsCons Most accurate picture of intervention effectiveness Time consuming Can be done any time after interventions have begun Requires several prerequisites: Decision rules Regularly graphed data Easily accessible intervention group student lists Allows you to examine effectiveness of individual intervention programs

56. Activity: Examining Intervention Groups (Decision Rules) Choose one (or if you have time you can do more ): –easyCBM –DIBELS Next –Your own data, if available Use the cups to indicate your progress: Practice Time! GreenYellowRed DoneStill WorkingNeed Help

57. On a scale of 1 to 5 how well do you understand the Examining Intervention Groups (Decision Rules) method? Do you have the prerequisites in place? What pieces would you need to build? Who would do this work? With a neighbor 12345 I’m completely lost I have a vague understanding I know it but need some practice I know it and could confidently do it I could teach others how to do it

58. 1.Summary of Effectiveness Reports 2.Comparing Growth Rates 3.Examining Intervention Groups (Decision Rules) Which option do you prefer? Why? How can you get started with this?

59. What can we do when we know our interventions are not effective?

61. 61 Why aren’t our interventions working? What can we do to improve them? Instruction:Curriculum: Environment:Learner (Instructor): Have we trained on intervention programs? Have we trained on instructional strategies? Have we provided on-going coaching with feedback? Research-based programs? Do we have enough materials? Instructional match of interventions to student need? Allocating enough time for interventions? Group sizes too large? Behavior management issues? Too many interventions? Other things to consider? FIDELITY

62. 30-60 minutes of isolated intervention time is not enough. Our most at-risk learners need the most coordinated instruction and support in generalizing skills across their day. Writing Specials ELD Core Reading Open Lunch & Recess Social Studies/Science Math Intervention 45 min 4 hours 15 min 1 hour

63. Are we placing kids appropriately? –Intervention Placement Are we monitoring their progress and making changes appropriately? –Intervention Group Review (20% Meetings) This afternoon

64. How effective are you interventions? Roughly what percentage of students make adequate progress? Use one of the discussed methods to determine the actual percentage of students that are making adequate progress in your interventions Homework!