1. Basic FBA to BSP Using FBA to Develop Function- Based Support for Students with Mild to Moderate Problem Behavior Module 7: Evaluation

2. 1. Define the Problem Behavior 2. Conduct assessment for behavior support planning a. Functional Behavioral Assessment Defining behavior in observable & measureable terms Ask staff and student about where, when, & why behavior occurs See the behavior during specified routines Hypothesize a final summary of where, when, & why behavior occurs 3. Design an individualized behavior support plan (BSP) Ensure technical adequacy Ensure contextual fit 4. Ensure Fidelity of Implementation 5. Monitor Plan Impact on Student Behavior Adapt BSP and implementation as needed based on on-going monitoring The Basic FBA to BSP Process Adapted from Horner, Albin, Todd, Newton & Sprague, 2011

3. Basic FBA to BSP Training Series Module 1- Teaching Basic Principles Module 2- FBA: Practice Interviewing Module 3- FBA: Practice Observing Module 4- Critical Features of BSP Module 5- Building BSP from FBA Module 6- Implementation & Evaluation Module 7- Evaluation

4. 4 BasicComplex For:Students with mild to moderate problem behaviors (behaviors that are NOT dangerous or occurring in many settings) Students with moderate to severe behavioral problems; may be dangerous and/or occurring in many settings What:Relatively Simple and Efficient process for behavior support planning based on “practical” FBA data Time-intensive process that involves emergency planning, family- centered planning, and collaboration with outside agencies Developed by whom: Team of school-based professionals (e.g., PBS team members whose job responsibilities include FBA and behavior support planning) School-based team including professionals trained to develop and implement intensive interventions for students with severe problem behaviors (e.g., behavior specialist) Basic vs. Complex FBA/BSP Focus of this training series

5. Objectives By the end of this module you will be able to: 1.Define the necessary components of evaluation plans and provide examples of appropriate short- and long-term goals 2.Describe data collection procedures that would be used to track implementation fidelity and student progress when provided with a sample BSP 3.Describe the process for conducting and products that should result from a Plan Review Meeting 5

6. 6 Review #1 The Basic FBA to BSP training series is designed to teach school staff to conduct assessment and develop supports for students with what type of behavior? –Mild to Moderate, NON-DANGEROUS Behaviors

7. 7 Review #2 What are the 4 steps of the Basic FBA Process (Hint: D.A.S.H)? #1: Define behavior in observable, measureable terms #2: Ask staff about When, Where, & Why the behavior occurs #3: See the behavior (direct observation) #4: Hypothesize a final summary of when, where and why behavior occurs

8. 8 Review #3 Please list the 4 critical components of Behavior Support Plans: #1: Competing Behavior Pathway #2: Prevention, Teaching, and Consequence Strategies #3: Implementation Plan #4: Evaluation Plan

9. 9 Evaluation Planning Student Outcomes & Plan Fidelity

10. Evaluation Planning Setting Goals – Student Outcomes & Implementation Fidelity Measurement – Data Collection methods must be Feasible & Accurate Regular Progress Monitoring & Decision Making

11. 11 Evaluation Planning The team leader will ensure that the BSP includes an evaluation plan with measures of: – Student Outcomes With a short-term goal and long-term goal – Plan Implementation And a specific date the team will next meet to review progress

12. The team identifies: - Short-term goal - Long-term goal - Specific evaluation procedures - Date to meet and evaluate the effectiveness of the plan BSP Evaluation Planning Form

13. Evaluation Planning Developing and Writing Goals

14. EVALUATION PLAN Behavioral Goal (Use specific, observable, measurable descriptions of goal) What is the short-term behavioral goal? _________ Expected date What is the long-term behavioral goal? _________ Expected date Data to be Collected Procedures for Data CollectionPerson Responsible Timeline Is Plan Being Implemented? Is Plan Making a Difference? Plan date for review meeting (suggested within 2 weeks) ________________ The team identifies: - Short-term goal - Long-term goal - Specific evaluation procedures - Date to meet and evaluate the effectiveness of the plan BSP Evaluation Planning Form E valuation Procedures

15. Developing & Writing Goals Components of a GOAL – Condition – Student – Behavior – Criterion – Measurement Plan 15

16. Use Competing Behavior Pathway to Guide Goal Development Goal Framework During, when (CONDITION), (STUDENT) will (BEHAVIOR) at least of the time (CRITERION) as measured by (MEASURE)

17. Developing Goals Short-term BSP Goal During Reading, when asked to complete independent writing tasks, (Condition) Jonas (Student) will work in his seat quietly OR appropriately request peer help (Behavior) at least 70% of the time (Criterion) as measured by ratings on a daily point card (Measurement). 17

18. Using the Competing Behavior Pathway to guide Goal Development 18 GOAL: – Condition use information about the Routine & Antecedents – Student name – Behavior – Criterion – Measurement Plan

19. will as measured by. GOAL CONDITION & STUDENT During, when ………………….., Use Competing Behavior Pathway to Guide Goal Development Continue to next task Escape independent writing task Completes work independently Talk to peers and make loud noises (e.g. animal sounds) Raise hand to request peer help When asked to complete independent writing tasks No Setting Event identified Routine: READING Short-term goal Student: JONAS JONAS READING, asked to complete independent writing tasks

20. will as measured by. GOAL CONDITION & STUDENT During, when ………………….., Use Competing Behavior Pathway to Guide Goal Development Continue to next task Escape difficult writing task Completes work independently Throwing materials & cursing Raise hand to request break or easier task Difficult writing tasks (multiple paragraph essay) No Setting Event identified Routine: WRITING Short-term goal Student: LEROY LEROY WRITING, asked to write a multiple paragraph essay

21. Developing Goals Behavior, Criteria & Measure Goals should: A. Be written in observable, measureable terms What specific behaviors will you increase/decrease? – Reduce Problem Behavior – Increase use of Replacement Behavior – Increase APPROXIMATIONS of the Desired Behavior B. Include specific mastery criteria How will you know when the student has met the goal? 21

22. Using the Competing Behavior Pathway to guide Goal Development 22 GOAL: – Condition – Student name – Behavior – short term or long-term goal? – Criterion – measurable, numeric – Measurement Plan – feasible data collection plan

23. 23 Measuring Student Outcomes The team leader will ensure that the BSP includes an evaluation plan with: – A short-term goal that is reasonable based on current performance Focused on measuring decreasing problem behavior and increasing Replacement behavior Usually about a 2 week goal – A long-term goal focused on increasing desired behavior A longer range goal (e.g. 3 months, 6 months, end of the year)

24. Use Competing Behavior Pathway to Guide Goal Development Typical Consequence Maintaining Consequence Desired Behavior Problem Behavior Alternate Behavior Antecedent Setting Event Routine: Long-term goal Short-term goal

25. Evaluation Planning: Short- term Goals Short-term goal – Focus on reducing problem behavior and increasing student’s use of the identified Replacement behavior &

26. Measurement & Criterion Primary focus of measurement in Short-term Goal = reduction in problem behavior – The outcome that is likely to be most meaningful to the teachers and staff members involved… – An important focus for maintaining buy-in and implementation!

27. Measuring Student Behavior Considerations: – Does the measure capture the specific tasks/target behaviors of interest? Was it a “good” or “bad” day? vs. How many talk-outs occurred during Spanish class today? – Is the measure sensitive enough to change? Are we tracking specific student behaviors?

28. Linking Goals with Intervention An emphasis of PBIS intervention is a focus on teaching kids what to do rather than what not to do – Prompt Desired Behavior on the Point Card Reframe our language from Reducing Problem Behavior to Desired Behavior We can say the same thing in different ways – Instead of What NOT to do focus on What to do: What NOT to do: Out of seat and ripping up assignments Focus on What to do: In seat and respecting class materials

29. Activity #1 Reframing Language Problem Behavior Disruptive, talking to peers Hitting, poking and grabbing peers Yelling and calling staff names (e.g. ogre, weirdo) doodling, staring out the window, off task What to Do:

30. Use Competing Behavior Pathway to Guide Goal Development Continue to next task Escape independent writing task Completes work independently Talk to peers and make loud noises (e.g. animal sounds) Raise hand to request peer help When asked to complete independent writing tasks No Setting Event identified Routine: READING Short-term goal Student: JONAS SHORT TERM GOAL: During Reading, when asked to complete independent writing tasks, Jonas will (BEHAVIOR) …….. OR …… WORK QUIETLY IN HIS SEAT (reduction in problem behavior) RAISE HAND TO REQUEST PEER HELP (Alternate Behavior)

31. Developing Goals Short-term Goal During Reading, when asked to complete independent writing tasks, (Condition) Jonas (Student) will work in his seat quietly OR appropriately request peer help (Behavior) at least 70% of the time (Criterion) as measured by ratings on a daily point card (Measurement). Notice that the goal is stated positively (what to do) rather than negatively… reductions of problem behavior can be measured by monitoring the expected behavior 31 Measuring Reduced Problem Behavior via increased Positive Behavior

32. Evaluation Planning: Short-Term Goals Short-term goal – Approximation of the desired behavior

33. Short Term Goals for Leroy Behavioral Goals ( Always include mastery criteria ) What is the short-term behavioral goal? During Writing, when asked to write a multiple paragraph essay Leroy will respect class materials and use respectful language OR ask appropriately for an easier task or for a “break” from difficult tasks at least 75% of the time as measured by a daily point card for 2 consecutive weeks. Approximation of the Desired Behavior Leroy will complete at least 40% of his assigned work in his Writing class. __2/12____ Expected date During Writing class, Leroy is currently engaging in problem behavior (throwing materials and cursing) to escape difficult writing tasks approximately 4 days per week. On average, he is completing only 25-30% of his work in class. Short-term goal Increase Alt. Behavior & Reduce Problem Behavior + Approximation toward Desired Behavior? Short-term goal Encourage Approximations of the Desired Behavior without throwing materials or cursing same

34. Leroy will respect class materials and use respectful language OR ask appropriately for an easier task or “break” from difficult tasks at least 75% of the time as measured by a daily point card for 2 consecutive weeks. He will also complete at least 40% of assigned work Leroy will respect class materials and use respectful language at least 80% of the time while asking for a “break” no more than 3 times per day as measured by daily point card. He will also complete at least 60% of assigned work. Leroy will respect class materials and use respectful language at least 80% of the time while asking for a “break” no more than 3 times per day as measured by daily point card. He will also complete at least 80% of assigned work. Short-Term Goal Long-Term Goal Leroy will complete at least 90% of his assigned work with no need for breaks and no more than 3 incidents of problem behavior (throwing materials, cursing) for one month. Intermediate Goals: Approximations

35. Evaluation Planning: Long-term Goals Long-term goal – Focus on desired behavior & sustained reductions in problem behavior Begin by reinforcing approximations of desired behavior

36. Use Competing Behavior Pathway to Guide Goal Development Continue to next task Escape independent writing task Completes work independently Talk to peers and make loud noises (e.g. animal sounds) Raise hand to request peer help When asked to complete independent writing tasks No Setting Event identified Routine: READING Student: JONAS LONG TERM GOAL (Desired Behavior): During Reading, when asked to complete independent writing tasks, Jonas will (BEHAVIOR) …….. Work Quietly, Independently Completing 90% of Assigned Work Long-term goal

37. Example Goals for Leroy Behavioral Goals ( Always include mastery criteria ) Long-Term Goal During Writing, when asked to write a multiple paragraph essay Leroy will complete at least 90% of his assigned work with no need for breaks and no more than 3 incidents of problem behavior (throwing materials, cursing) for one month as measured by Incident Reports. __5/1____ Expected date During Writing class, Leroy is currently engaging in problem behavior (throwing materials and cursing) to escape difficult writing tasks approximately 4 days per week. On average, he is completing only 25-30% of his work in class. Short-term goal Increase Alt. Behavior & Reduce Problem Behavior + Approximation toward Desired Behavior? Long-term goal Increase Desired Behavior & Reduce Problem Behavior without throwing materials or cursing same

38. Sample Short-Term Goal for Dexter Short-term: Dexter will: a) stay on task without leaving his seat or talking to peers about unrelated topics for at least 75% of independent work time, OR b) appropriately ask to work with a peer (or work independently) in Writing, c) complete at least 25% of his daily writing assignments for 4 out of 5 days across 2 consecutive weeks. 38 Increase use of Replacement Behavior Decrease Problem Behavior Increase Approximations of Desired Behavior Mastery Criteria

39. Measurement Plan

40. Measurement Plan Feasible Data Collection For efficiency and flexibility in the Basic FBA to BSP process we encourage use of a point card for data collection. We will provide a generic point card, but if there is a point card in your school for Check- In/Check-Out we encourage you to align the Basic FBA to BSP point card template to the same format

41. #1 - Reduction in Problem Behavior #2 - Use of Alt Beh or Reduction in Problem Behavior #3 - Desired Behavior or Approximations Measuring the Short-term Goal Example Point Card Routine: __________________

42. #1 – Quiet and in Seat #2 – Appropriately Raises Hand to Request Peers Help Measuring the Short-term Goal Example Point Card Short Term Goals Routine: ___Reading________ JONAS Approx of Desired Beh #3 On Task

43. Criterion Short-Term Goal – Use baseline data to develop a REASONABLE, ACHIEVABLE initial goal for the student – Initial success is important for buy-in of the student & staff – Link the Goal with the Incentive/Reinforcement Intervention in the BSP – We Suggest Starting with the Goal no lower than 70%... If student is not meeting the goal, revise the BSP and increase supports

44. Criterion Short Term Goal – Remember Reasonable Goals or Expectation include both the Expected Behavior & Timeframe Start with shorter intervals for feedback (e.g. 10 minute intervals v. 60 minute intervals) Don’t hold incentives to the end of the week – The Short term goal will CONTINUOUSLY be revised as student moves closer to achieving the long-term goal

45. #1 – Quiet and in Seat #2 – Appropriately Raises Hand to Request Peers Help Measuring the Short-term Goal Example Point Card Short Term Goals Routine: ___Reading________ JONAS 7028 Reasonable Behavioral Expectations & Criterion? Reasonable TimeFrames? -Length of the feedback intervals? -10 min. intervals in 60 min. Rdg block Reasonable TimeFrames? Frequency of Access to Incentive? -daily incentive if meets goal in Reading #3 On Task Approx of Desired Beh

46. Is the Plan Making a Difference? FAQ: I see the student every day, why do I need to collect data? Answer: Data help us to Document what has occurred and the variables responsible Predict future performance Be accountable for our own behavior Determine when program modifications are needed

47. Short Term Goals for Leroy Behavioral Goals ( Always include mastery criteria ) What is the short-term behavioral goal? During Writing, when asked to write a multiple paragraph essay Leroy will respect class materials and use respectful language OR ask appropriately for an easier task or for a “break” from difficult tasks at least 75% of the time as measured by a daily point card for 2 consecutive weeks. Approximation of the Desired Behavior Leroy will complete at least 40% of his assigned work in his Writing class. __2/12____ Expected date During Writing class, Leroy is currently engaging in problem behavior (throwing materials and cursing) to escape difficult writing tasks approximately 4 days per week. On average, he is completing only 25-30% of his work in class. Short-term goal Increase Alt. Behavior & Reduce Problem Behavior + Approximation toward Desired Behavior? Short-term goal Encourage Approximations of the Desired Behavior without throwing materials or cursing same

48. Use Respectful Language Use materials appropriately LeRoy Feb. 4 th Measure Decrease in Problem Behavior through focus on increasing Expected Behaviors Measuring the Short-term Goal During Writing, when asked to write a multiple paragraph essay Leroy will respect class materials and use respectful language OR ask appropriately for an easier task or for a “break” from difficult tasks at least 75% of the time as measured by a daily point card for 2 consecutive weeks. Leroy will complete at least 40% of his assigned work in his Writing class. On Task

49. Activity #2 (page 89) When asked to read aloud or answer questions during small group reading lessons, Charlie makes inappropriate comments (e.g., “This is so stupid”), puts her head down on the desk and refuses to comply with any of the teachers directions. The FBA shows that this problem behavior is maintained by adult attention, and the team has decided to teach Charlie to raise her hand and ask appropriately for teacher help/attention. Develop: a) a short-term goal, b) an intermediate goal, and c) a long-term goal for Charlie. Make sure that you include: – observable, measureable descriptions of behavior – mastery criteria

50. Evaluating the BSP: Role of Plan Implementers Plan Implementers – Collect fidelity of implementation data at least 1 x per week Report any difficulties in implementing the plan to the team leader – Collect data on student behavior at least 3 x per week to assess progress If problem behavior increases or escalates contact team leader immediately

51. Is the Plan Being Implemented? Considerations When Developing Measures of Fidelity: Does the measure capture the specific tasks/target behaviors of interest? – Is the plan being implemented? Did I implement the plan? vs. Did I check in with student and provide specific praise when she entered class?

52. Consequence/Function Teacher reprimands, teacher takes to time-out Access Teacher / Adult Attention Antecedent Teacher teaching to whole class or praising another student Setting Event None Identified Replacement Behv Raise hand for adult attention or to answer a question Problem Behavior Making noises with mouth or hands, leaning on or grabbing others or materials, out of seat Consequence Feeling of success, adult praise for appropriate behavior Routine: Circle Time on Carpet Setting EventsManipulate AntecedentTeach BehaviorAlter Consequences Eliminate/Neutralize Setting Events N/A Eliminate/Modify Antecedents - Sheldon will sit in front of teacher / tape outline on carpet marking his spot -Weighted vest -Frequent adult attention/ stickers for appropriate behavior (at least 1 x per min) Prompt Alt/Des Behavior - Pre-teach on the way to circle time (model raising hand, “safe body”) Teach Replacement Behavior Teach Sheldon to raise hand for attention Teach what it means to have a “listening ears” Teach Desired Behavior/Skills Teach appropriate sitting on carpet (use examples/non- examples; teach outside of circle time routine) Rewards Sheldon will earn stickers and teacher praise for sitting appropriately (5 stickers will = teacher recognition in front of class) Teacher will quickly respond to and praise hand raising Response to Problem At first sign of problem behavior, remind Sheldon to raise his hand using visual cue only (minimize adult attention to problem behavior) Time out (no longer than 3 min), praise and bring back after several seconds of approp behv Desired Behavior Sitting appropriately and responding with group during circle time Example: Sheldon

53. Questions MTWTHF 1. Was the tape outline on the carpet marking Sheldon’s spot? 0 1 2 2. Was Sheldon given a weighted vest at the beginning of circle time? 0 1 2 3. Did I remind Sheldon what appropriate sitting/raising hand looks like at the beginning of circle time? 0 1 2 4. Did I provide stickers for hand raising, and staying in seat? 0 1 2 5. Did I minimize attention to problem behavior? 0 1 2 6. Did I provide frequent attention for appropriate behavior? 0 1 2 7. If Sheldon had to go to time-out, did I allow him to rejoin the group within 2 minutes? 0 1 2 TOTALS 0 1 2 Example: Implementer Checklist 0 = Did Not Implement; 1 = Partial Implementation; 2 = Full Implementation

54. Measures: Implementation Fidelity DateUsed White Board Gave reminders and prompts Allowed BreaksChecked in and out 0 1 2 0 1 2 0 1 2 0 1 2 0 1 2 0 1 2 0 1 2 0 1 2 0 1 2 0 1 2 0 1 2 0 1 2 Total Points = ________________ Points Possible = 24 Today _________________% Goal _________________% Example tool for daily collection of implementation fidelity data. Implementation Fidelity

55. Evaluation Procedures for Maizey Data to be CollectedProcedures for Data Collection Person Responsible Timeline (Implement as of 2/27) Is plan being implemented? Implementation Checklist Develop Checklist & Disseminate to Implementers Complete Checklist Collect Checklist at end of day Enter Data into Graph Mrs. Foster Mr. T Mrs. Foster Today Daily Today/ Daily Daily Is plan making a difference? Daily point card data % assignments completed Develop Point Card Track Data Daily -point card & % completion Enter data into Graph Mrs. Foster Mr. T Mrs. Foster Today Daily Plan Review Date 2/14/15

56. Activity #3 (page 91) Using the summary statement and strategies provided for Kelly on pages 91-92 in your guide, work with a partner to determine/describe: What SPECIFIC student behaviors will you collect data on?

57. Graphing Data for Decision Making

59. Meeting to Review the Plan

60. The plan is a WORKING DOCUMENT! Team members meet regularly to: a) Monitor progress b) Modify the plan as needed to: Make the plan more effective or Change mastery criteria and increase student independence

61. Review Meeting: Role of the Team Leader Guide team in reviewing DATA for each component of the plan to document: 1. Is each strategy being implemented as designed? 2. Is the plan resulting in change in student behavior? 3. Do data indicate that the plan needs to be modified and how? 4. What is the date of the next Review Meeting?

62. 7 8 The Implementation Plan is used to record: : 3/21/11 1. The extent to which the plan is being implemented and… 2. Team evaluation decisions made, based on the data presented at the meeting Add more multi-digit problems Monitor Completed/ Discontinue Monitor

63. Barriers to Implementation Questions to ask if plan is not being implemented: – Do implementers understand how and when to use strategies? – Have implementers been provided with a way to measure implementation? – Are strategies feasible in the natural setting? – Are there ways that plan can be modified to make implementation more likely? *Note: If the plan is not being implemented with fidelity, we can not assess if the plan is working.

64. The Plan is Working! Now What?? The student is making progress, but has not yet met the short term goal. – Continue plan and progress monitoring The student has met the short-term goal!! – Create a new short-term goal that more closely approximates the desired behavior and ASK: Can we expand the plan/implement in more contexts? Can we being fading antecedent prompts and modifications? Should we start fading and/or modifying rewards? Are there strategies that we can use to increase the student’s monitoring of his or her own behavior?

65. Key Points An EVALUATION PLAN for determining A) if the plan is being implemented B) if the plan is making a difference in student behavior C) when team members will meet again to discuss progress Both the Team Leader and Team Implementers collect Fidelity and Effectiveness data regularly. The Behavior Support Plan is a Work in Progress!!! Team members meet every two weeks to determine: – Is the BSP being implemented as agreed on by the team? – Is the student making progress? – Do we need to modify the plan to: Improve effectiveness Increase student independence? – When is the date of our next meeting?

66. Check #1 When developing a BSP evaluation plan, short-term goals should focus on increasing the student’s use of: The Replacement Behavior The long-term goal focuses on increasing: Desired Behavior/Skills 66

67. Edgar’s team has met to review his progress since implementing the BSP. The data show that Edgar’s problem behavior has not decreased in the past 2 weeks. What is the first question that Edgar’s team should ask? Is the plan being implemented with fidelity?? Check #2

68. Check #3 Go to page 105 in your guidebook. In teams of 3, please select a sample scenario and use the forms provided in your guidebook (pages 106-108) to build a complete student BSP. 68

69. 69 At the end of “free-choice” time, when asked to transition back to her desk, Charlie verbally refuses, cries, and falls to the floor to avoid transitioning to a less preferred activity. This is most likely to occur on days when Charlie does not take her medicine before school. Charlie’s “tantrums” occur 3-4 times per week and can last up to 10 minutes. Example #1: Charlie

70. 70 During large-group instruction in Math, when students are asked to attend to the materials being projected on the screen at the front of the class. Garrett often turns around in his chair, gets out of his seat and walks around the room, and makes comments to or faces at peers. FBA data show that his behavior is mostly likely maintained by peer attention. Garrett is currently off-task approximately 85% of the time during large-group and he is turning in less than 50% of his math assignments. Example #2: Garret

71. Task Over the next two weeks… Please meet with team members at your school to develop an Evaluation Plan based on FBA/BSP results…. And hold a BSP Review Meeting Your role as Team Leader will be to guide the team through the questions on pages 101 and 102 in your workbook and to ensure that the BSP is: – Complete (don’t forget about implementation and evaluation plans!) – Function-Based – Contextually Relevant 71

72. Comments/Questions about Module 7 At the bottom of page 109 please write any comments/questions you may have pertaining to this module. Thank you for your time & attention! 72