1. Response to Intervention www.interventioncentral.org Monitoring Student Progress at the Secondary Level Jim Wright www.interventioncentral.org

2. Response to Intervention www.interventioncentral.org 2 “Everybody is entitled to their own opinion but they’re not entitled to their own facts. The data is the data.” Dr. Maria Spiropulu, Physicist New York Times, 30 September 2003 (D. Overbye) Other dimensions? She’s in pursuit. F1, F4

3. Response to Intervention www.interventioncentral.org 3 “Few agree on an appropriate curriculum for secondary students…; thus it is difficult to determine in what areas student [academic] progress should be measured.” -- Espin & Tindal (1998) Source: Espin, C. A., & Tindal, G. (1998). Curriculum-based measurement for secondary students. In M. R. Shinn (Ed.) Advanced applications of curriculum-based measurement. New York: Guilford Press.

4. Response to Intervention www.interventioncentral.org 4 RTI: Research Questions Q: What RTI Identification Method Will Best Determine What Students Are ‘Responders’ or ‘Non-Responders’ to Intervention? There are several methods in the research literature to determine ‘non-responders’ to intervention (e.g., dual discrepancy, slope discrepancy). What is the ‘best’ method to reliably differentiate students who do or do not respond to RTI interventions? Source: Fuchs, D., & Deshler, D. D. (2007). What we need to know about responsiveness to intervention (and shouldn’t be afraid to ask).. Learning Disabilities Research & Practice, 22(2), 129–136.

5. Response to Intervention www.interventioncentral.org 5 Secondary Students: Should Interventions Be ‘Off- Level’ or Focus on Grade-Level Academics? There is a lack of consensus about how to address the academic needs of students with deficits in basic skills in secondary grades (Espin & Tindal, 1998). –Should the student be placed in remedial instruction at a point of ‘instructional match’ to address those basic-skill deficits? (Instruction is adjusted down to the student) –Or is time better spent providing the student with compensatory strategies to learn grade-level content and ‘work around’ those basic-skill deficits? (Student is brought up to current instruction) Source: Espin, C. A., & Tindal, G. (1998). Curriculum-based measurement for secondary students. In M. R. Shinn (Ed.) Advanced applications of curriculum-based measurement. New York: Guilford Press.

6. Response to Intervention www.interventioncentral.org 6 K123456789101112 Rdng Fluency Rdng-Basic Comprehension Subject-Area Rdng Comprehension Remediating Academic Deficits: The Widening Curriculum Gap… Rdng-Basic Comprehension Reading Fluency Widening academic gap (middle school). Student is significantly off-level. The building curriculum barely overlaps the student’s point of ‘instructional match’. Reading Fluency Small academic gap (elementary school). Student is only mildly off- level. The building curriculum overlaps the student’s point of ‘instructional match’. Largest academic gap (high school). Student is significantly off-level. The building curriculum does not overlap the student’s point of ‘instructional match’ at all. Rdng-Basic Comprehension

7. Response to Intervention www.interventioncentral.org 7 Measuring General vs. Specific Academic Outcomes General Outcome Measures: Track the student’s increasing proficiency on general curriculum goals such as reading fluency. An example is CBM-Oral Reading Fluency (Hintz et al., 2006). Specific Sub-Skill Mastery Measures: Track short-term student academic progress with clear criteria for mastery. An example is CBA-Math Computation Fluency (Burns & Gibbons, 2008). Sources: Burns, M. K., & Gibbons, K. A. (2008). Implementing response-to-intervention in elementary and secondary schools: Procedures to assure scientific-based practices. New York: Routledge. Hintz, J. M., Christ, T. J., & Methe, S. A. (2006). Curriculum-based assessment. Psychology in the Schools, 43, 45-56.

8. Response to Intervention www.interventioncentral.org Common Methods for Monitoring Student Progress Toward Behavioral & Academic Goals

9. Response to Intervention www.interventioncentral.org 9 RTI Team Initial Meeting Form: Secondary Student Progress- Monitoring Page 15

10. Response to Intervention www.interventioncentral.org 10 RTI Team Initial Meeting Form: Secondary Student Progress- Monitoring Page 16

11. Response to Intervention www.interventioncentral.org 11 RTI Team Initial Meeting Form: Secondary Student Progress- Monitoring Page 17

12. Response to Intervention www.interventioncentral.org Making Use of Existing (‘Extant’) Data

13. Response to Intervention www.interventioncentral.org 13 Extant (Existing) Data (Chafouleas et al., 2007) Definition: Information that is collected by schools as a matter of course. Extant data comes in two forms: –Performance summaries (e.g., class grades, teacher summary comments on report cards, state test scores). –Student work products (e.g., research papers, math homework, PowerPoint presentation). Source: Chafouleas, S., Riley-Tillman, T.C., & Sugai, G. (2007). School-based behavioral assessment: Informing intervention and instruction. New York: Guilford Press.

14. Response to Intervention www.interventioncentral.org 14 Summative data is static information that provides a fixed ‘snapshot’ of the student’s academic performance or behaviors at a particular point in time. School records are one source of data that is often summative in nature—frequently referred to as archival data. Attendance data and office disciplinary referrals are two examples of archival records, data that is routinely collected on all students. In contrast to archival data, background information is collected specifically on the target student. Examples of background information are teacher interviews and student interest surveys, each of which can shed light on a student’s academic or behavioral strengths and weaknesses. Like archival data, background information is usually summative, providing a measurement of the student at a single point in time.

15. Response to Intervention www.interventioncentral.org 15 Formative assessment measures are those that can be administered or collected frequently—for example, on a weekly or even daily basis. These measures provide a flow of regularly updated information (progress monitoring) about the student’s progress in the identified area(s) of academic or behavioral concern. Formative data provide a ‘moving picture’ of the student; the data unfold through time to tell the story of that student’s response to various classroom instructional and behavior management strategies. Examples of measures that provide formative data are Curriculum- Based Measurement probes in oral reading fluency and Daily Behavior Report Cards.

16. Response to Intervention www.interventioncentral.org 16 Advantages of Using Extant Data (Chafouleas et al., 2007) Information is already existing and easy to access. Students are less likely to show ‘reactive’ effects when data is collected, as the information collected is part of the normal routine of schools. Extant data is ‘relevant’ to school data consumers (such as classroom teachers, administrators, and members of problem-solving teams). Source: Chafouleas, S., Riley-Tillman, T.C., & Sugai, G. (2007). School-based behavioral assessment: Informing intervention and instruction. New York: Guilford Press.

17. Response to Intervention www.interventioncentral.org 17 Drawbacks of Using Extant Data (Chafouleas et al., 2007) Time is required to collate and summarize the data (e.g., summarizing a week’s worth of disciplinary office referrals). The data may be limited and not reveal the full dimension of the student’s presenting problem(s). There is no guarantee that school staff are consistent and accurate in how they collect the data (e.g., grading policies can vary across classrooms; instructors may have differing expectations regarding what types of assignments are given a formal grade; standards may fluctuate across teachers for filling out disciplinary referrals). Little research has been done on the ‘psychometric adequacy’ of extant data sources. Source: Chafouleas, S., Riley-Tillman, T.C., & Sugai, G. (2007). School-based behavioral assessment: Informing intervention and instruction. New York: Guilford Press.

18. Response to Intervention www.interventioncentral.org 18 ‘Elbow Group’ Activity: What Data Should Be Collected for RTI Team Meetings? What are the ‘essential’ sources of archival data that you would like collected and brought to every RTI Problem-Solving Team meeting?

19. Response to Intervention www.interventioncentral.org Grades as a Classroom-Based ‘Pulse’ Measure of Academic Performance

20. Response to Intervention www.interventioncentral.org 20 Grades & Other Teacher Performance Summary Data (Chafouleas et al., 2007) Teacher test and quiz grades can be useful as a supplemental method for monitoring the impact of student behavioral interventions. Other data about student academic performance (e.g., homework completion, homework grades, etc.) can also be tracked and graphed to judge intervention effectiveness. Source: Chafouleas, S., Riley-Tillman, T.C., & Sugai, G. (2007). School-based behavioral assessment: Informing intervention and instruction. New York: Guilford Press.

21. Response to Intervention www.interventioncentral.org 21 2-Wk4-Wk6-Wk8-Wk10-Wk12-Wk 9/23/0710/07/0710/21/0711/03/0711/20/0712/05/07 Marc Ripley (From Chafouleas et al., 2007) Source: Chafouleas, S., Riley-Tillman, T.C., & Sugai, G. (2007). School-based behavioral assessment: Informing intervention and instruction. New York: Guilford Press.

22. Response to Intervention www.interventioncentral.org 22 Online Grading Systems

23. Response to Intervention www.interventioncentral.org Assessing Basic Academic Skills: Curriculum-Based Measurement

24. Response to Intervention www.interventioncentral.org 24 Assessing Basic Academic Skills: Curriculum-Based Measurement Reading: These 3 measures all proved ‘adequate predictors’ of student performance on reading content tasks: –Reading aloud (Oral Reading Fluency): Passages from content-area tests: 1 minute. –Maze task (every 7 th item replaced with multiple choice/answer plus 2 distracters): Passages from content-area texts: 2 minutes. –Vocabulary matching: 10 vocabulary items and 12 definitions (including 2 distracters): 10 minutes. Source: Espin, C. A., & Tindal, G. (1998). Curriculum-based measurement for secondary students. In M. R. Shinn (Ed.) Advanced applications of curriculum-based measurement. New York: Guilford Press.

25. Response to Intervention www.interventioncentral.org 25 Assessing Basic Academic Skills: Curriculum-Based Measurement Mathematics: Single-skill basic arithmetic combinations an ‘adequate measure of performance’ for low-achieving middle school students. Websites to create CBM math computation probes: Source: Espin, C. A., & Tindal, G. (1998). Curriculum-based measurement for secondary students. In M. R. Shinn (Ed.) Advanced applications of curriculum-based measurement. New York: Guilford Press. www.interventioncentral.org www.superkids.com

26. Response to Intervention www.interventioncentral.org 26 Assessing Basic Academic Skills: Curriculum-Based Measurement Writing: CBM/ Word Sequence is a ‘valid indicator of general writing proficiency’. It evaluates units of writing and their relation to one another. Successive pairs of ‘writing units’ make up each word sequence. The mechanics and conventions of each word sequence must be correct for the student to receive credit for that sequence. CBM/ Word Sequence is the most comprehensive CBM writing measure. Source: Espin, C. A., & Tindal, G. (1998). Curriculum-based measurement for secondary students. In M. R. Shinn (Ed.) Advanced applications of curriculum-based measurement. New York: Guilford Press.

27. Response to Intervention www.interventioncentral.org 27 A Note About Monitoring Behaviors Through Academic Measures… Academic measures (e.g., grades, CBM data) can be useful as part of the progress-monitoring ‘portfolio’ of data collected on a student because: Students with problem behaviors often struggle academically, so tracking academics as a target is justified in its own right. Improved academic performance generally correlates with reduced behavioral problems. Individualized interventions for misbehaving students frequently contain academic components (as the behavior problems can emerge in response to chronic academic deficits). Academic progress-monitoring data helps the school to track the effectiveness of the academic interventions.

28. Response to Intervention www.interventioncentral.org Breaking Down Complex Academic Goals into Simpler Sub-Tasks: Discrete Categorization

29. Response to Intervention www.interventioncentral.org 29 Identifying and Measuring Complex Academic Problems at the Middle and High School Level Students at the secondary level can present with a range of concerns that interfere with academic success. One frequent challenge for these students is the need to reduce complex global academic goals into discrete sub-skills that can be individually measured and tracked over time.

30. Response to Intervention www.interventioncentral.org 30 Discrete Categorization: A Strategy for Assessing Complex, Multi-Step Student Academic Tasks Definition of Discrete Categorization: ‘Listing a number of behaviors and checking off whether they were performed.’ (Kazdin, 1989, p. 59). Approach allows educators to define a larger ‘behavioral’ goal for a student and to break that goal down into sub-tasks. (Each sub- task should be defined in such a way that it can be scored as ‘successfully accomplished’ or ‘not accomplished’.) The constituent behaviors that make up the larger behavioral goal need not be directly related to each other. For example, ‘completed homework’ may include as sub-tasks ‘wrote down homework assignment correctly’ and ‘created a work plan before starting homework’ Source: Kazdin, A. E. (1989). Behavior modification in applied settings (4 th ed.). Pacific Gove, CA: Brooks/Cole..

31. Response to Intervention www.interventioncentral.org 31 Discrete Categorization Example: Math Study Skills General Academic Goal: Improve Tina’s Math Study Skills The student Tina:  Approached the teacher at the end of class for a copy of class note.  Checked her daily math notes for completeness against a set of teacher notes in 5 th period study hall.  Reviewed her math notes in 5 th period study hall.  Started her math homework in 5 th period study hall.  Used a highlighter and ‘margin notes’ to mark questions or areas of confusion in her notes or on the daily assignment.  Entered into her ‘homework log’ the amount of time spent that evening doing homework and noted any questions or areas of confusion.  Stopped by the math teacher’s classroom during help periods (T & Th only) to ask highlighted questions (or to verify that Tina understood that week’s instructional content) and to review the homework log.

32. Response to Intervention www.interventioncentral.org 32 Discrete Categorization Example: Math Study Skills Academic Goal: Improve Tina’s Math Study Skills General measures of the success of this intervention include (1) rate of homework completion and (2) quiz & test grades. To measure treatment fidelity ( Tina’s follow-through with sub-tasks of the checklist), the following strategies are used :  Approached the teacher for copy of class notes. Teacher observation.  Checked her daily math notes for completeness; reviewed math notes, started math homework in 5 th period study hall. Student work products; random spot check by study hall supervisor.  Used a highlighter and ‘margin notes’ to mark questions or areas of confusion in her notes or on the daily assignment. Review of notes by teacher during T/Th drop-in period.  Entered into her ‘homework log’ the amount of time spent that evening doing homework and noted any questions or areas of confusion. Log reviewed by teacher during T/Th drop-in period.  Stopped by the math teacher’s classroom during help periods (T & Th only) to ask highlighted questions (or to verify that Tina understood that week’s instructional content). Teacher observation; student sign-in.

33. Response to Intervention www.interventioncentral.org RTI: Additional Assessment Resources

34. Response to Intervention www.interventioncentral.org Student Independent Work (‘Permanent Products’): Assessing Completion, Accuracy and Overall Quality

35. Response to Intervention www.interventioncentral.org 35 Steps in Assessing Student Independent Work: pp. 69-70

36. Response to Intervention www.interventioncentral.org 36 Hypotheses for Poor or Limited Work Completion pp. 70-71 Student ScenariosSample Intervention Ideas The student completes independent work quickly with time to spare--but the work contains ‘careless’ mistakes or is of poor quality.  Provide the student with incentives to slow down and use the full time allocated to complete the assignment.  Require that the student use a quality checklist or rubric to review work before turning it in. If the student attempts to turn in completed work that does not meet teacher expectations, send the student back to his or her seat to continue to work on the assignment. The student was off-task during much of the work session. The assignment was not completed within the time allocated.  Use strategies to increase the student’s attention to task (e.g., teacher redirection to task, student self-monitoring of work completion). The completed assignment was of poor quality and/or contained many errors.  Review with the student the skills or strategies required for the assignment.  Give the student correctly completed models similar to what the student must produce for the assignment. Encourage the student to refer to these models whenever he or she is ‘stuck’.  Approach the student in a low-key manner periodically during independent seatwork to see if the student requires assistance.  Provide the student an incentive (e.g., five additional minutes of free time) if the student improves the quality or accuracy of the work. The student did not complete the assignment in the allotted time. However, the student demonstrated a high degree of quality and/or accuracy in his or her work. Boost the student’s speed by providing him or her with opportunities to practice the skills or strategies required for the assignment. Give the student feedback and encouragement as the student increases his or her working speed.

37. Response to Intervention www.interventioncentral.org 37 Independent Seatwork Observation Form p.72

38. Response to Intervention www.interventioncentral.org Instructional Setting Rating Sheet

39. Response to Intervention www.interventioncentral.org 39 Instructional Setting Rating Sheet p. 73

40. Response to Intervention www.interventioncentral.org How Do We Know Whether Motivation is a Barrier to Learning?: Student Motivation Assessment

41. Response to Intervention www.interventioncentral.org 41 Schoolwork Motivation Assessment p. 6 Sources: Witt, J., & Beck, R. (1999). One minure academic functional assessment andinterventions: "Can't" do it…or "won't" do it? Longmont, CO: Sopris West. Witt, J. C., VanDerHeyden, A. M., Gilbertson, D. (2004). Troubleshooting behavioral interventions: A systematic process for finding and eliminating problems. School Psychology Review, 33, 363-381.

42. Response to Intervention www.interventioncentral.org 42 Schoolwork Motivation Assessment p. 7

43. Response to Intervention www.interventioncentral.org 43 Schoolwork Motivation Assessment Step 1: Assemble an incentive menu Step 2: Create two versions of a timed worksheet Step 3: Administer the first timed worksheet to the student WITHOUT incentives. Step 4: Compute an improvement goal. 5: Have the student select an incentive for improved performance. Step 6: Administer the second timed worksheet to the student WITH incentives. Step 7: Interpret the results of the academic motivation assessment to select appropriate interventions.

44. Response to Intervention www.interventioncentral.org 44 Schoolwork Motivation Assessment Step 1: Assemble an Incentive menu Create a 4-5 item menu of modest incentives or rewards that students in the class are most likely to find motivating.

45. Response to Intervention www.interventioncentral.org 45 Schoolwork Motivation Assessment Step 2: Create two versions of a timed worksheet Make up two versions of custom student worksheets. The worksheets should be at the same level of difficulty, but each worksheet should have different items or content to avoid a practice effect. NOTE: If possible, the worksheets should contain standardized short-answer items (e.g., matching vocabulary words to their definitions) to allow you to calculate the student’s rate of work completion.

46. Response to Intervention www.interventioncentral.org 46 Schoolwork Motivation Assessment Step 3: Administer the first timed worksheet to the student WITHOUT incentives. In a quiet, non-distracting location, administer the first worksheet or CBM probe under timed, standardized conditions. Collect the probe or worksheet and score.

47. Response to Intervention www.interventioncentral.org 47 Schoolwork Motivation Assessment Step 4: Compute an improvement goal. After you have scored the first CBM probe or worksheet, compute a ’20 percent improvement goal’. Multiply the student’s score on the worksheet by 1.2. This product represents the student’s minimum goal for improvement. Example: A student who completed 20 correct items on a timed worksheet will have an improvement goal of 24 (20 x 1.2 = 24).

48. Response to Intervention www.interventioncentral.org 48 Schoolwork Motivation Assessment Step 5: Have the student select an incentive for improved performance. Tell the student that if he or she can attain a score on the second worksheet that meets or exceeds your goal for improvement (Step 3), the student can earn an incentive. Show the student the reward menu. Ask the student to select the incentive that he or she will earn if the student makes or exceeds the goal.

49. Response to Intervention www.interventioncentral.org 49 Schoolwork Motivation Assessment Step 6: Administer the second timed worksheet to the student WITH incentives. Give the student the second CBM probe. Collect and score. If the student meets or exceeds the pre-set improvement goal, award the student the incentive.

50. Response to Intervention www.interventioncentral.org 50 Schoolwork Motivation Assessment Step 7: Interpret the results of the academic motivation assessment to select appropriate interventions. ACADEMIC INTERVENTIONS ONLY. If the student fails to meet or exceed the improvement goal, an academic intervention should be selected to teach the appropriate skills or to provide the student with drill and practice opportunities to build fluency in the targeted academic area(s).

51. Response to Intervention www.interventioncentral.org 51 Schoolwork Motivation Assessment Step 7(Cont): Interpret the results of the academic motivation assessment to select appropriate interventions. COMBINED ACADEMIC AND PERFORMANCE INTERVENTIONS. If the student meets or exceeds the improvement goal but continues to function significantly below the level of classmates, an intervention should be tailored that includes strategies to both improve academic performance and to increase the student’s work motivation.

52. Response to Intervention www.interventioncentral.org 52 Schoolwork Motivation Assessment Step 7(Cont): Interpret the results of the academic motivation assessment to select appropriate interventions. PERFORMANCE INTERVENTIONS ONLY. If the student meets or exceeds the improvement goal with an incentive and shows academic skills that fall within the range of ‘typical’ classmates, the intervention should target only student work performance or motivation.

53. Response to Intervention www.interventioncentral.org 53 ‘Motivation Assessment in Advanced Subject Areas’ Activity Brief behavior analysis of motivation (e.g., Schoolwork Motivation Assessment ) is most effective for basic skill areas. In your ‘elbow groups’: Discuss ways that RTI Teams could collect information about whether motivation is an ‘academic blocker’ on more advanced academic tasks (e.g., writing a term paper) or subject areas (e.g., trigonometry).

54. Response to Intervention www.interventioncentral.org RTI Teams: Recommendations for Data Collection

55. Response to Intervention www.interventioncentral.org 55 RTI Teams: Recommendations for Data Collection Collect a standard set of background information on each student referred to the RTI Team. RTI Teams should develop a standard package of background (archival) information to be collected prior to the initial problem-solving meeting. For each referred student, a Team might elect to gather attendance data, office disciplinary referrals for the current year, and the most recent state assessment results.

56. Response to Intervention www.interventioncentral.org 56 RTI Teams: Recommendations for Data Collection For each area of concern, select at least two progress-monitoring measures. RTI Teams can place greater confidence in their progress-monitoring data when they select at least two measures to track any area of student concern (Gresham, 1983)-ideally from at least two different sources (e.g., Campbell & Fiske, 1959). With a minimum of two methods in place to monitor a student concern, each measure serves as a check on the other. If the results are in agreement, the Team has greater assurance that it can trust the data. If the measures do not agree with one another, however, the Team can investigate further to determine the reason(s) for the apparent discrepancy.

57. Response to Intervention www.interventioncentral.org 57 RTI Teams: Recommendations for Data Collection Monitor student progress frequently. Progress-monitoring data should reveal in weeks--not months--whether an intervention is working because no teacher wants to waste time implementing an intervention that is not successful. When progress monitoring is done frequently (e.g., weekly), the data can be charted to reveal more quickly whether the student’s current intervention plan is effective. Curriculum-based measurement, Daily Behavior Report Cards, and classroom observations of student behavior are several assessment methods that can be carried out frequently.