Team Workshop for Tier 3 Advanced SW-PBS Teams: Session 3 – Monitoring Fidelity and Professional Development MO SW-PBS Training

Working Agreements Be Respectful Be Responsible Be a Problem Solver Be an active listener—open to new ideas Use notes for side bar conversations Be Responsible Be on time for sessions Silence cell phones—reply appropriately Be a Problem Solver Follow the decision making process Work toward consensus and support decisions of the group “These will be the Working Agreements we will honor during all our training sessions this year.”

Attention Signal Practice When I raise my hand and ask for “ATTENTION HERE PLEASE” finish your conversation within 10 seconds and wait for the next instruction “One of our agreements is to follow the attention signal.” “Your team will work with your school to develop an attention signal you will use in every setting.” “In order to get all of you focused after discussions, activities or breaks, I will ________________________________________.” (Insert your attention signal here.) “Your task will be to finish your sentence, quiet your voice and ____________________________.” (Insert what you want participants to do.) MO SW-PBS

Introductions Insert an Introductions Activity. Choose an Introductions Activity from SharePoint>Coaches Curriculum>Tier 1 Training>Standard Activities>Introduction Activities

Outcomes Today At the end of today’s session, you will be able to… Document a plan for monitoring fidelity of Tier 3 intervention implementation. Review Professional Development of Tier 3 systems and practices Content learned in Day 2: Assess social validity (Day 2) Evaluate impact of strategies (Day 2) MO SW-PBS

Identifying Data Collection Needs Questions to Ask Example Data Source What data do we need to make good decisions? (Student File Measures) What are we going to collect/track, and how are we going to collect it? (Measure Description) How often will we collect data? (Time Segments, Data Entry Schedule) How will we collect and report data? (Metric) Documentation: Individualized Education Plan (IEP) Behavior Support Plan (BSP) Data Collection Sheet(s) Functional Behavioral Assessment (FBA) Documentation FACTS/interviews, direct observation data, student assessments Additional Resources: Student Support Team District Tier II/III Coach These are some needs that your school may have regarding data collection

Review Activity: Define the behavior observed and how you could progress monitor this behavior On what schedule? Using what method? Behaviors: Off task behavior, disrespect, disruptions, non-compliance Have groups define one of the behavior listed below – teams should identify how often they would observe the behavior to progress monitor and identify which method would best work to monitor this data

Fidelity of Implementation Measures How well did we do what we said we would do? Fidelity measures the degree to which the intervention was implemented as defined/expected. Example: Staff will strive for 80% fidelity of implementation as measured weekly (or bi-weekly) on a scale of 1-5. Make it easy for staff to record! Fidelity Check Board: X on number line in staff room Fist to Five: During team meeting Fidelity Check Basket: Drop off your response Direct observation (requires trusting & supportive staff climate) Self Check (Staff checks off steps completed) Fidelity of implementation is a new concept for many schools and will require extra support to identify contextually appropriate options for collecting and using this data.

Introducing Fidelity of Implementation Fidelity data is an opportunity to discuss the student’s support plan, not a tool to pick on staff! Questions to ask of fidelity data: Is the plan being implemented? Is the plan a good contextual fit for the environment? Is additional training or coaching needed? Are there sufficient resources allocated to implement the plan? Has something changed (e.g., staff, schedule, student behaviors)? Questions teams should ask to ensure fidelity before moving forward with plan changes

Is the plan having an effect on the student’s behavior? Outcome Measures Is the plan having an effect on the student’s behavior? “Outcome Measures” track what effect the implementation of the plan is having on the student’s academic and/or social behavior. What do we want to track and how? Problem behavior (e.g., duration, frequency, intensity) Replacement/Desired behavior (e.g., duration, frequency, points earned) Academic behavior (e.g., correct words per minute, test scores, correct sounds produced, level of participation) Skill acquisition (e.g., steps of hand-washing completed, vocabulary words) Questions teams should ask to ensure that the plan is having an impact

Data Collection Samples Here are some sample data collection tools. One of your recommended take-home activities is to use these tools to practice setting up measures in your facilitator account.

Fidelity of Implementation Scale of 1 (low) – 5 (high) With what level of quality did you implement the plan? Wk 1: 1 2 3 4 5 Wk 2: 1 2 3 4 5 Wk 3: 1 2 3 4 5 Wk 4: 1 2 3 4 5 Wk 5: 1 2 3 4 5 Wk 6: 1 2 3 4 5 Wk 7: 1 2 3 4 5 How many days did you implement the plan? Wk 1: 1 2 3 4 5 Wk 2: 1 2 3 4 5 Wk 3: 1 2 3 4 5 Wk 4: 1 2 3 4 5 Wk 5: 1 2 3 4 5 Wk 6: 1 2 3 4 5 Wk 7: 1 2 3 4 5 One example of a way to collect data on fidelity

Staff Fidelity Card Another fidelity collection example

Student Point Card A way of incorporating fidelity on a CICO card

Combined Data Collection Sheets Student: B.B. Week: ­last week Count of Disrespectful Behavior Assignments Fidelity (1-5) (Fridays only) Completed Assigned Monday 18 1 10   Tuesday 20 Wednesday 17 11 Thursday Absent - Friday 19 9 3 Combined form of behavior data, academic data, and fidelity data

Activity: Using a BIP from your school – determine which steps of the BIP could be assessed for fidelity and create a chart to track and monitor fidelity data Have schools use the BIP they brought to the training to develop a plan for tracking fidelity

Action Plan Goal: Establish a system for developing an evaluation and monitoring plan Action Steps: Develop procedures for assessing fidelity of implementation of BIP Have teams turn to the Develop a System for Providing Ongoing Professional Development page (244) of their Tier 3 Action Plan and give them time to plan how they will address these action steps/activities and plan any other action steps/activities they wish to add. Tier 3 Action Plan MO SW-PBS 244

Individual Student Information System (ISIS-SWIS) ISIS-SWIS is a decision system for students receiving more intensive, individualized supports for academic, social, or mental health services. Teams using ISIS-SWIS can: Upload and store documentation. Define data collection measures. Summarize data for decision making. ISIS-SWIS is a system for storing data and team decision making

What is ISIS-SWIS? Individual Student Information System ISIS-SWIS is an application within the SWIS Suite designed to coordinate and monitor individualized student support. ISIS-SWIS allows teams to: Set up, collect, and monitor a student’s outcome data on individualized goals based on team agreements. Set up, collect, and monitor fidelity data about the implementation of the support plan. Upload and store critical student plan documents needed for planning and decision making. Summarize outcome and fidelity data for problem solving and decision making. Benefits of incorporating ISIS-SIWS

Where Does ISIS-SWIS Fit? ISIS-SWIS is a decision system for students requiring more intensive and individualized supports for academic social or mental health services. CICO-SWIS is a decision system for targeted or group-based interventions for students needing additional support beyond the Universal or Tier 1 system. ISIS-SWIS is the third application in the SWIS Suite. Each of the three SWIS applications (SWIS; CICO-SWIS and ISIS-SWIS) is designed to assist school personnel to make effective and efficient decisions, and together the three applications follow the multi-tiered prevention model advocated within PBIS and RtI efforts. Tier 1: The School-Wide Information System (SWIS) is a web-based decision system designed to help school personnel to use office referral data to monitor progress of school-wide and individual student interventions. Tier 2: CICO-SWIS is a decision system for targeted or group-based interventions for students needing additional support beyond the Universal or Tier 1 system. Tier III: ISIS-SWIS is a decision system for students requiring more intensive and individualized supports in academics, social or mental health services. * Note that this approach assumes that support at one level builds upon additional tiers of support. Students needing Tier III support are receiving all 3 tiers (not just individualized). School-Wide Information System (SWIS) is a web-based decision system designed to help school personnel to use office referral data to monitor progress of school-wide and individual student interventions.

Advantages of ISIS-SWIS Efficiency Structured creation and maintenance of student files One home for progress monitoring, goal setting, and decision making Instantaneous access to data Equity Equal access to quality support plan management Enabling of clear roles, responsibilities, and predictability Quality Supports compliance with federal procedures for Tier III support Comprehensive student file for quality decision making Documentation of progress and intervention history Flexibility Files and measures tailored to a student’s needs Efficiency: All relevant information (e.g., support plans, FBA’s, meeting minutes, grades, background information) in one place to improve meetings, support timely decision making, promote regular review of data (recommended twice weekly). Teams can easily access and assess intervention fidelity and effectiveness to identify specific changes needed (vs. starting from scratch or making large, unnecessary changes). Equity: Evaluation of fidelity as well as student outcome data ensures that the student receives the right amount and type of support needed to access education. Quality: ISIS-SWIS allows teams to better organize documentation and easily access data in a usable format to more quickly see where and when changes to the support plan are needed. Flexibility: The measures, documents, data collection schedule, and reports are all set up based on the needs of the student as identified by the student’s support team. Number of measures (fidelity and outcome) Team member access to data Measure details and metric types Data collection schedule(s) Individualized goal(s) Wide variety of reporting options History of plan changes and anecdotal notes

Efficiency ISIS-SWIS provides structure & tools for creating and maintaining student support files. ISIS-SWIS brings individualized student progress monitoring, goal setting, and decision making into one place. ISIS-SWIS supports efficient data entry to keep records current. ISIS-SWIS allows teams to easily access data for timely review and decision making. Efficiency: All relevant information (e.g. support plans, FBA’s, meeting minutes, grades, background information) in one place to improve meetings, support timely decision making, promote regular review of data (recommended twice weekly). Teams can easily access and assess intervention fidelity and effectiveness to identify specific changes needed (vs. starting from scratch or making large, unnecessary changes).

Equity ISIS-SWIS provides equal access to quality support for students requiring individualized interventions. ISIS-SWIS provides staff with predictability, leading to clear roles and responsibilities. Equity: Evaluation of fidelity as well as student outcome data ensures that the student receives the right amount and type of support needed to access education.

Quality ISIS-SWIS brings together goals/objectives from a student’s support plan (e.g., FBA, BSP, IEP) into a comprehensive student file to allow for quality decision making. ISIS-SWIS supports compliance with federal procedures around Tier III Support (e.g., FBA, BSP, IEP). ISIS-SWIS documents the history of student interventions and progress. Quality: ISIS-SWIS allows teams to better organize documentation and easily access data in a usable format to more quickly see where and when changes to the support plan are needed.

Flexibility ISIS-SWIS allows progress monitoring to be individualized based solely on the team’s identification of the student’s needs. Number of measures (fidelity and outcome) Team member access to data Measure details and metric types Data collection schedule(s) Individualized goal(s) Wide variety of reporting options History of plan changes and anecdotal notes Flexibility: The measures, documents, data collection schedule, and reports are all set up based on the needs of the student as identified by the student’s support team

Why was ISIS-SWIS developed? Schools needed: A way to efficiently enter, store, and summarize data for decision making. A system designed for team-based planning. A system with the flexibility to monitor progress for students with a variety of behavioral and academic needs and supports. Developers: May, S., Talmadge, N., Todd, A. W., Horner, R. H., McGovern, S., Morris, J., Conley, K., Sampson, N., Eliason, B., & Cave, M. Revise the developer/acknowledgements when available.

Why was ISIS-SWIS developed? Overall Goal: Make schools more effective learning environments. Strategy: Repeatedly giving people the right information, at the right time, in the right format is the single most effective way to improve decision making and valued outcomes. (Gilbert, 1978) Behavior: Social behavior is the single most common reason students are excluded from schools. Trainer Note: Researchers and practitioners were working with schools and did not have a systematic way to collect, organize and analyze data for decision making. They were relying on creating their own unique spreadsheets and data tables to analyze data and evaluate effectiveness. The goal in creating SWIS was to make schools more effective learning environments by ensuring that they could readily access the information they needed to make decisions. Creation of SWIS was about providing people the right information, at the right time, in the right format to make the best possible decisions. The focus is on social-emotional learning & behavior because social behavior is the single most common reason that students are excluded from the learning environment (e.g., classrooms, schools).

Example of the student file checklist available in ISIS-SWIS

This is an example of the Metrics set up in ISIS-SWIS

Discussion Review with your team the ISIS-SWIS information. What is your team currently using to track data? As you continue your Tier 3 training, would ISIS-SWIS be a program your school might consider using? How can your team use this information to improve your Tier 3 implementation? Give teams time to discuss and share. MO SW-PBS

Outcomes Today At the end of today’s session, you will be able to… Document a plan for monitoring fidelity of Tier 3 intervention implementation. Review Professional Development of Tier 3 systems and practices Content learned in Day 2: Assess social validity (Day 2) Evaluate impact of strategies (Day 2) MO SW-PBS

Effective Professional Development Key to success will be the effectiveness of the staff development you provide. An effective intervention is one thing. Effective implementation of an effective intervention is a very different thing. Dean Fixsen, 2008 Providing quality professional development to your staff will be important in ensuring that your Tier 3 interventions are implemented with fidelity. MO SW-PBS

Effective Professional Development Effective professional development addresses the flaws of traditional approaches, which are often criticized for being fragmented, unproductive, inefficient, unrelated to practice, and lacking in intensity and follow-up. Eric Clearinghouse on Teaching and Teacher Education Can participants relate to these criticisms? MO SW-PBS

Reflect on Your Own Experience Make a t-chart for your team and label one side “Effective PD” and the other “Ineffective PD” What are the characteristics of effective professional development you have received? What are the characteristics of ineffective professional development you have received? List these characteristics on your team’s t-chart Once teams have completed their t-charts, have them share with the group. Are there common characteristics that a majority of groups mentioned for effective and ineffective professional development? Each team’s challenge is to provide effective professional development to their school staff on the systems, data, and practices for Tier 3 intervention. MO SW-PBS

Effective Professional Development Research-based Consistent, ongoing Convenient Relevant Differentiated Nobori, 2011 Research-based – let research guide your strategies (SW-PBS is research-based, including the Tier 3 intervention that you will be implementing – FBA/BIP) Consistent – create a long-term plan for PD and be consistent in the implementation across multiple years (The professional development you provide your staff will not be one-time at the beginning of the year before school starts. Staff development needs to be on-going, including new learning as well as “booster” sessions for review. Your staff will need to be trained so they can develop the FBA/BIP expertise needed to be contributing members to Action Teams. Additionally, staff will need to receive training in order to implement BIP strategies and collect data. Convenient – schedule at a time that increases the likelihood of attendance (When will this be for your school? Your team will need to work closely with your administrator to ensure that professional development is a priority) Relevant – focus PD on enhancing your school’s instructional model (Teachers will see a direct benefit in the improved behavior and positive relationships of the students participating in a Tier 3 intervention) Differentiated – provide choices when possible. i.e. an instructional approach and a technology approach (Does all professional development in your school need to be face-to-face, large group? Is using technology such as podcasts and webinars an option? What are other possibilities?) MO SW-PBS

Quality Professional Development Other characteristics: Includes training, practice, feedback and coaching or other follow-up procedures/support Provides opportunities for teachers to interact with peers Ismat, 1996 This is a partial list from the Eric Clearinghouse on Teaching and Teacher Education. Did the teams identify a lot of the characteristics that were on these two slides? You will see the impact of including training, practice, feedback, and coaching on the next slide. MO SW-PBS

How Did You Do? Compare the "Effective PD" column of the t-chart your team created earlier with the characteristics described by Norobi and Ismat. Are there similarities? Give teams time to compare and discuss. MO SW-PBS

Demonstrate Knowledge Demonstrate New Skill in Training    OUTCOMES (% of Participants who: Demonstrate Knowledge, Demonstrate New Skills in a Training Setting, and Use New Skills in the Classroom) Training Components Demonstrate Knowledge Demonstrate New Skill in Training Use New Skills in Classroom Theory and Discussion 10% 5% 0% Plus Demonstration in Training 30% 20% Plus Practice and Feedback in Training 60% Plus Coaching in the Classroom 95% After an analysis of more than 200 studies of in-service training methods, Joyce and Showers (1980) found that when the goal of training is related to the integration and transfer of complex ideas or requires significant modifications to teaching method, only the combination of all five components--theory, demonstration, practice, feedback, and coaching--was consistent with the transfer of training to the classroom (Joyce & Showers, 2002). After a few decades of research on training teachers, Joyce & Showers (2002) began to think of training and coaching as one continuous set of operations designed to produce actual changes in the classroom behavior of teachers. One without the other is insufficient. Now is the time for your team to start thinking how you can provide Tier 3 training to your staff that includes the components identified by Joyce and Showers. 187 Joyce and Showers, 2002

Discussion Discuss the findings of Joyce and Showers and how this information impacts the type of professional development you will need to provide your staff. Give teams time to compare and discuss. Ask: What role does your school counselor play in your Tier 3 PD system? According to the recent document created by a partnership between MSCA, DESE and SW-PBS entitled: “The Role of the Professional School Counselor in Implementing SW-PBS” (October 2013), The Professional School Counselor plays an integral role in all three SW-PBS Tier Teams as a resource by contributing professional knowledge to the development, implementation, and evaluation of the Tier procedures and interventions; promoting greater continuity between tiers; and being trained at each tier to fully understand and contribute to the success of SW-PBS. The counselor can share information and expertise in areas that align with the unique role and training of the Professional School Counselor, such as the science of behavior; confidentiality guidelines, regulations and rationale; and training in how to conduct observations. MO SW-PBS 187

Finding Time . . . . Can staff meetings have an instructional focus instead of a managerial focus? Can portions of grade level/departmental planning times be used for staff development? Can classes/activities be covered by other available school personnel for teachers to attend staff development? Can technology be utilized to allow for flexible training times? Other ideas? Each school is unique in schedules, budgetary restrictions for substitutes, coverage availability from other staff, and contractual/negotiated agreement restrictions. All schools also have competing initiatives. Teams can inventory those initiatives – are all of them current and necessary? (Many times schools continue to do something because “it’s always been that way” even though it is no longer effective or needed.) Do some of the initiatives have overlapping purposes and can be streamlined? Have teams share ideas that have worked for them in getting more time for staff development. Teams will need to work closely with their administrators in order to develop a professional development calendar. MO SW-PBS

Discussion What options are available in your school for scheduling Tier 3 professional development opportunities? How far in advance is the professional development calendar established? Do you need to start planning now to get additional time on the calendar for next year? Give teams time to discuss. MO SW-PBS

Updating Staff Develop standard procedures for providing updates to administrator, team, staff and others involved with student support What will be communicated? Who will communicate this information? How will it be communicated? When will it be communicated? These four questions will assist your team in developing your communication system; if you can answer these questions, your system for updating staff is in place. MO SW-PBS

Discussion Does your Tier 3 team have an effective system already established for updating staff? If not: How do your Tier 1 and Tier 2 teams provide regular staff updates? If that system is effective, can your Tier 3 team incorporate the same system? Give teams time to discuss. Ask them to share effective systems already in place. MO SW-PBS

Action Plan Goal: Develop a system for providing ongoing professional development Action Steps: Develop a system for regularly updating staff Create a calendar of ongoing professional development regarding Tier 3 Have teams turn to the Develop a System for Providing Ongoing Professional Development page (244) of their Tier 3 Action Plan and give them time to plan how they will address these action steps/activities and plan any other action steps/activities they wish to add. Tier 3 Action Plan MO SW-PBS 244

Tier 3 Staff Handbook Your Tier 3 staff handbook will serve as a reference for staff and document your Tier 3 data, systems, and practices Information in your Tier 3 Staff Handbook will provide the basis for your Tier 3 professional development As you go through this year, you will be developing your Tier 3 staff handbook; this handbook will include information that will be content for the professional development you provide your staff. The handout provides guidance for what you will include in your handbook as you develop your systems, data, and practices. Tier 3 Staff Handbook Organizer MO SW-PBS 200

Discussion Review the Tier 3 Staff Handbook Organizer. What do you already have available that can be added to your Tier 3 Staff Handbook? As you continue your Tier 3 training, who will be responsible for adding/updating your Tier 3 Staff Handbook? How will staff access the handbook? Will you provide hard copies? Electronic copies? Give teams time to discuss and share. Tier 3 Staff Handbook Organizer MO SW-PBS 200

Action Plan Goal: Develop a system for providing ongoing professional development Action Steps: Develop- a Tier 3 Staff Handbook Have teams turn to the Develop a System for Providing Ongoing Professional Development page (244) of their Tier 3 Action Plan and give them time to plan how they will address these action steps/activities and plan any other action steps/activities they wish to add. Tier 3 Action Plan MO SW-PBS

Next Session: T3A Session 4 Using Environmental Interventions March 30, 2016 Edplus Building Annex Things to Bring: Laptop, Tier 3 workbook, Action plan, Items needing to be worked on MO SW-PBS

Contact Information: Dr. Deb Childs childsde@Missouri.edu 636/373-1533