1. SST3 May, 2014

2. Overview 1. Progress Monitoring Assessment: What? Why? 2. Sample PM tools 3. Graphing Data 4. Data-Base Decisions Setting Goals Performance Level Rate of Learning 5. Linking cases to RTI processes

3. Assessment in a RTI model Benchmarking To screen and identify students who are at-risk and in need of interventions All students Three times a year All areas At grade-level Progress Monitoring To monitor progress of individual students and determine rate of improvement and need for adaptation of intervention Students who are not achieving benchmarks (PLP, IEP) Weekly, biweekly, monthly assessments In area of need At instructional level

4. What assessments do you use? ReadingMathSocial-Emotional Behavior What measures do you use? Who do you assess? When do you assess? How is the information used? Benchmark/ Progress Monitoring

5. Benefits of Progress Monitoring Parents and students know what is expected Teachers know what is working or not working with their instruction based on data Easy to understand way to show parents progress Teams have comprehensive data on student performance for decision making

6. Curriculum Based Measures - CBM Are assessments to monitor progress “indicators” important Are designed to serve as “indicators” of general achievement. CBM doesn’t measure everything, but measures the important things. standardized tests standard way. Are standardized tests to be given, scored, and interpreted in a standard way. researched Are researched with respect to psychometric properties to ensure accurate measures of learning. sensitiveshort periods Are sensitive to improvement in short periods of time. Are designed as brief measures that can be administered frequently. linked to decision making Are linked to decision making for promoting positive achievement and Problem-Solving

7. Tools National Center on Student Progress Monitoring www.studentprogress.org

8. Samples of CBMs Reading Math Writing Spelling

10. MAZE - CBM AIMSweb Reading Comprehension Measure www.aimsweb.com

11. DIBELS Phoneme Segmentation Fluency https://dibels.uoregon.edu

12. Taken from Fuchs, L. S., Hamlett, C. A., & Fuchs, D. (1998). Monitoring Basic Skills Progress: Basic Math Computation (2nd ed.). [computer program]. Austin, TX: ProEd. Available: from http://www.pro edinc.com MATH COMPUTATION

13. Concepts and Applications Sample page from a three- page test for Grade 2 Math Concepts and Applications From Monitoring Basic Skills Progress

14. CBM - Writing www.interventioncentral.org Total Words Written Correct Word Sequences Words Correctly Spelled

15. Spelling AIMSweb Spelling Probes

18. Graphing Graphing is an essential part of PM Without graphic displays, the decision making process is difficult Teacher graphing vs. Student graphing

19. Graphing Data How to Develop Graphs : VW Beetle vs. SUV vs. Race Car Hand Graphing Excel and Chart Dog Web-based data systems

20. Hand Graphing 1. Establish Baseline (Median score) 2. Set up graph 3. Set Goal 4. Draw Aimline 5. Measure Student Progress 6. Plot Student Performance 7. Connect Indicators of Student Performance 8. Analyze Student Performance 9. Make Instructional Changes 10. Continue to Measure and Monitor Student Performance

21. Hand Graphing

22. Testing Sessions BaselineSession 1 Session 3 Session 2 Session 4 Session 6 Session 5 Session 7Session 8 Number of Words Read Correctly 30 35 40 45 50

23. Hand Graphing Advantages Easy to do No technology required Students can easily maintain their own graphs Can be done immediately Free Disadvantages Added paper Organization required No long-term storage Not automatic

24. Excel Templates Graphing made easy: Practical tools for school psychologists http://www.oswego.edu/~mcdougal /web_site_4_11_2005/index.html Academic Monitoring Behavior Monitoring

25. Chart Dog www.interventioncentral.org

26. Excel Templates and Chart Dog Advantage Automatic Storage capability built- in Easy to do Clear displays of data Free if you have EXCEL, Chart Dog is free Disadvantage Requires technology Time to enter data Students may not be able to do data entry themselves Requires some understanding of EXCEL or Chart Dog

27. Web Based System

28. Web-Based System Advantage Web based data entry from anywhere Storage capability built-in Trend line drawn automatically Can annotate graphs interventions/goals Norms –benchmarks and Rate of Improvement Lots of flexibility Email graphs Disadvantage Requires technology Cost Students may not be able to do data entry themselves Requires some training

29. To do this will take new learning for everyone

30. Data-Based Decisions 1. Performance Level Gaps in Performance Below Grade Level Special Education Significant Discrepancy 2. Rate of Learning Trend in performance (slope) Response to Instruction General Direction, Rate of Change

31. 1.Performance Level: Gap/ Discrepancy Be objective. Does it refer to an observable/measurable characteristic of behavior? Use numbers to define the discrepancy. Percentile rank Discrepancy Ratios Cut scores Norms

32. Norms… What to use? Local, National Local norms can be helpful to determine local performance levels and rates of progress Time consuming and costly to develop National norms and research norms are available. BUT….

33. Percentile Ranks Requires a Larger Normative Data Base, Preferably Benchmark Data < 25th At Risk, Consider Problem-Solving at the Group Level <10th Potential Severe Problem, Consider Individual Problem Solving 1. Performance Level

34. Discrepancy Ratios 1. Sample 5-7 Students or Whole Class, Grade 2. Figure Median and Graph 3. Divide by 2 and Graph 4. Students Who Performance Below the Line May Need Problem Solving Performance Level

35. Can Compute… Peer Median Target Student Median 145 40 = Discrepancy of 3.6x

36. Cut Scores A number which represents the point at which scores can be divided into different groups for decision-making purposes. (E.g. does not meet, meets, and exceeds expectations) May be based on research (e.g., a correlation between scoring at or above a certain level on a CBM or DIBELS task and future academic success) or expectation (e.g., grades at C or above, no more than 3 office referrals). Performance Level

37. Enables team to make decisions about levels of support and resource from the start Generally speaking… A student who is 1.5x discrepant from his/her peers may benefit from intensive group interventions. A student who is 2-2.5x discrepant from his/her peers is appropriate for individualized problem-solving and intensive intervention resources may be appropriate. Example: Jessica is 2.1x discrepant from peers on the Math CBM and may benefit from intensive interventions in math. Data-base decisions on performance level.

38. Rebecca 2 nd grader List all areas of concern: Off-task behavior Reading difficulties Poor handwriting Identify primary area of concern and define it in observable and measurable terms: Reading Definition: number words read correctly when reading a grade level passage orally Collect baseline data on primary area of concern and state discrepancy statement: Baseline data collected in the area of test from CBM reading probes Discrepancy Statement: Rebecca reads 41 WRC per minute in Fall of 2nd grade while her peers read ____ WRC per minute ___________

39. Rob 7 th grader List all areas of concern: Calling out Lack of homework completion Poor handwriting Identify primary area of concern and define it in observable and measurable terms: Work Completion Definition: Turning in teacher assigned work at beginning of class period on the day that it is due. Collect baseline data on primary area of concern and state discrepancy statement: Baseline data collected in the area of review from teacher grade books Discrepancy Statement: Rob currently turns in homework 54% of the time while his peers turn in homework 86% of the time (_____ discrepant)

40. Data-Based Decisions 1. Performance Level Gaps in Performance PLP Not at Grade Level Special Education Significant Discrepancy 2. Rate of Learning Trend in performance (slope) Response to Instruction General Direction, Rate of Change

41. 2. Rate of Learning Why? Determine when what we are doing isn’t working and intervene early Better able to predict student success at meeting goals Better able to identify who needs more intensive instruction

42. NameGradeAreaInitial Performance Discrepancy Follow Up Performance Discrepancy Outcome Decision Bill3Reading10th percentile20th percentile Satisfactory; Maintain Intervention Susie2Math1.2xNANo Severe Problem Rob4Homewor k 65% completion64% completion No Progress, Problem solve and Modify intervention plan Rate of Learning Tracking Student Outcomes Using Initial Performance Discrepancies

43. Rate of Learning Analyzing Rate using PM Data Rules: Setting Goals Data Point Rules Trend Line Rules Slope

44. Setting Goals 1. End of the Year Benchmarks GLEs for Reading Fluency (2 nd grade 80-100 WPM, 5 th grade 125-150) AIMSweb Math Computation Norms ( 1 st grade 17 DPM, 5th grade 52 DPM) 2. National Norms for Improvement Math Calculations (>.3 DPM 2 nd and 3 rd grade, >.5 DPM 4-6 th grade) (Fuchs, 2006) Reading Fluency (Deno, 2005) GradeModestReasonableAmbitious 1-21 Word Per Week1.5 Word Per Week2.0 Word Per Week 3-6.5 Word Per Week1.0 Word Per Week1.5 Word Per Week

45. Setting Goals 3. Individual ROI Weekly rate of improvement in “baseline slope” calculated from 8 data points (Slope: Difference of highest and lowest/#weeks) Baseline multiplied by 1.5 Product multiplied by number of weeks until end of year Add to student’s final baseline score to produce end of year goal. Baseline Reading scores: 52, 54, 52, 53, 55, 58, 55, 56 Difference: 58-52 = 5 Divide by number of weeks: 5/8 =.625 (SLOPE) Baseline multiplied by 1.5:.625 × 1.5 =.9375 Number of weeks left (6 weeks):.9375 ×6 =5.6 Add to final baseline score: 56+5.6 = 61.6 End of the year goal 62

46. Decisions based on data-points Decisions are based on at least 4 data points If all 4 scores fall above goal-line, responding to instruction (increase goal if continues for 4 more data points) If scores are hovering about the goal line, continue what you are doing. If all 4 scores are below goal-line, but parallel, decide to “wait” for 4 more points to see if student performance accelerates in level to reach original goal. If all 4 scores fall below goal-line, not responding to instruction, revise plan and implement different teaching strategy. Mark change on graph with vertical line. Derived from: Fuchs and Fuchs (2006) and Shapiro (2006)

48. Decisions based on trend lines Trend lines based on 6-8 data-points If trend line is steeper than goal line, increase the goal. If trend line is flatter than goal line, revise instruction If trend line equals goal line, make no change at this time.

50. Data-base decision on slope Rate of Improvement = slope or r(in statistics) Y = slopeX + intercept Consider discrepancy from ROI norms

52. Is Jim responding to the intervention?

53. Checkpoint: Is Jim Responding to the Math Intervention? Are modifications necessary? End of Year: What would you say about Jim’s Math?

54. How can you use the literacy, math and social- emotional/behavior assessments that you have? Do they provide information about a student’s performance level rate of learning area of instructional need?

55. Your chart might include one of these literacy measures? R-CBMRigbyDRA Performance Level YESYES?YES Rate of Learning YESNO Instructional needs YES?YES

56. What is the difference between data-driven and data informing?

57. Rebecca- 2 nd grader Fall benchmark some concern 41 WPM, <25 th percentile, 1.96X Discrepant Instructional Plan small group guided reading book bag for home with leveled readers

58. Rebecca- 2 nd grader

59. What additional data is needed to modify interventions and meet Rebecca’s needs? Possible Hypotheses ReviewInterviewObserveTest Rebecca reads 41 WRC per minute in the Fall of 2nd grade while her peers read ____WRC per minute and with small group guided reading limited rate of improvement because …

60. Case Example: Rebecca Possible Hypotheses ReviewInterviewObserveTest She had insufficient opportunity to practice Interview Teacher/Parent Observe during SSR CBM She lacks the phonological skills to read the text fluently. Review her Records R-CBM scores Interview Teacher/Parent Phonological Assessment Instructional materials are above her reading level. Review her Records/CBM scores Interview Teacher/Parent CBM survey level assessment Rebecca reads 41 WRC per minute in the Fall of 2 nd grade while her peers read 80 WRC per minute and with small group guided reading limited rate of improvement because …

61. Case Example: Rebecca Predicted Hypotheses ReviewInterviewObserveTest She had insufficient opportunity to practice Instructional Planning Form revealed Rebecca had 15 minutes of independent reading practice per day Observation revealed that during SSR, Rebecca often played with items on desk. CBM probes indicated that she was not performing comparable to peers She lacks the phonological skills to read the text fluently. Review of records indicated that she had done well in remedial phonics program the previous year and was graduated out of the program. IPF suggested that instructional time was given to phonological skills. CBM probes indicated that she was not performing comparable to peers; DIBELS assessments did not point to a phonological awareness or phonics problem. Her reading level is not instructionally appropriate. Teacher reports showed that she was at the 25 th percentile in the Spring of 1 st grade Teacher felt that curriculum was appropriate Survey level assessment revealed that Rebecca had mastered 1 st grade text

62. Case Example: Rebecca Rebecca reads 41 WRC per minute in the Fall of 2 nd grade while her peers read 80 WRC per minute and is improving at a limited rate (.5 ROI) because … She had insufficient opportunity to practice.

63. What was the impact of progress monitoring assessments for Rebecca and her teachers?

64. Rebecca’s progress during Reader’s Theatre

65. We will have tough choices to make – we’ll decide based on what’s best for our kids

66. References Deno, S., Lembke, E. and Anderson, A. (2005) Progress Monitoring Study Group Content Module available at www.studentprogress.org. Fuchs and Fuchs (April, 2006) Progress Monitoring: Identifying LD and Improving Student Outcomes. Presentation at National SEA Conference on SLD Determination. Kansas City, MO. Hawkins, and Tilly, D. (Sept. 2005). Response to Intervention On the Ground: Diagnosing the Learning Enabled. Pacific Northwest Institute on Special Education and the Law. Shapiro, E. (2006). Advanced Issues in Monitoring Student Progress in Reading. Presentation for Pennsylvania Training and Technical Assistance Network on on RTI. Tilly (Sept, 2005) Problem Identification and Analysis. Presentation at the Principal Lead Problem Solving Initiative, Highland Park, IL.