1. Team Workshop for Tier 3 Advanced SW-PBS Teams: Session 3 – Monitoring Fidelity and Professional Development MO SW-PBS Training

2. Working Agreements Be Respectful Be Responsible Be a Problem Solver
Be an active listener—open to new ideas
Use notes for side bar conversations
Be Responsible
Be on time for sessions
Silence cell phones—reply appropriately
Be a Problem Solver
Follow the decision making process
Work toward consensus and support decisions of the group
“These will be the Working Agreements we will honor during all our training sessions this year.”

3. Attention Signal Practice
When I raise my hand and ask for “ATTENTION HERE PLEASE” finish your conversation within 10 seconds and wait for the next instruction
“One of our agreements is to follow the attention signal.”
“Your team will work with your school to develop an attention signal you will use in every setting.”
“In order to get all of you focused after discussions, activities or breaks,
I will ________________________________________.”
(Insert your attention signal here.)
“Your task will be to finish your sentence, quiet your voice and ____________________________.”
(Insert what you want participants to do.)
MO SW-PBS

4. Introductions Insert an Introductions Activity.
Choose an Introductions Activity from
SharePoint>Coaches Curriculum>Tier 1 Training>Standard Activities>Introduction Activities

5. Outcomes Today At the end of today’s session, you will be able to…
Document a plan for monitoring fidelity of Tier 3 intervention implementation.
Review Professional Development of Tier 3 systems and practices
Content learned in Day 2: Assess social validity (Day 2)
Evaluate impact of strategies (Day 2)
MO SW-PBS

6. Identifying Data Collection Needs
Questions to Ask
Example Data Source
What data do we need to make good decisions? (Student File Measures)
What are we going to collect/track, and how are we going to collect it? (Measure Description)
How often will we collect data? (Time Segments, Data Entry Schedule)
How will we collect and report data? (Metric)
Documentation:
Individualized Education Plan (IEP)
Behavior Support Plan (BSP)
Data Collection Sheet(s)
Functional Behavioral Assessment (FBA) Documentation
FACTS/interviews, direct observation data, student assessments
Additional Resources:
Student Support Team
District Tier II/III Coach
These are some needs that your school may have regarding data collection

7. Review Activity:
Define the behavior observed and how you could progress monitor this behavior
On what schedule?
Using what method?
Behaviors: Off task behavior, disrespect, disruptions, non-compliance
Have groups define one of the behavior listed below – teams should identify how often they would observe the behavior to progress monitor and identify which method would best work to monitor this data

8. Fidelity of Implementation Measures
How well did we do what we said we would do?
Fidelity measures the degree to which the intervention was implemented as defined/expected.
Example: Staff will strive for 80% fidelity of implementation as measured weekly (or bi-weekly) on a scale of 1-5.
Make it easy for staff to record!
Fidelity Check Board: X on number line in staff room
Fist to Five: During team meeting
Fidelity Check Basket: Drop off your response
Direct observation (requires trusting & supportive staff climate)
Self Check (Staff checks off steps completed)
Fidelity of implementation is a new concept for many schools and will require extra support to identify contextually appropriate options for collecting and using this data.

9. Introducing Fidelity of Implementation
Fidelity data is an opportunity to discuss the student’s support plan, not a tool to pick on staff!
Questions to ask of fidelity data:
Is the plan being implemented?
Is the plan a good contextual fit for the environment?
Is additional training or coaching needed?
Are there sufficient resources allocated to implement the plan?
Has something changed (e.g., staff, schedule, student behaviors)?
Questions teams should ask to ensure fidelity before moving forward with plan changes

10. Is the plan having an effect on the student’s behavior?
Outcome Measures
Is the plan having an effect on the student’s behavior?
“Outcome Measures” track what effect the implementation of the plan is having on the student’s academic and/or social behavior.
What do we want to track and how?
Problem behavior (e.g., duration, frequency, intensity)
Replacement/Desired behavior (e.g., duration, frequency, points earned)
Academic behavior (e.g., correct words per minute, test scores, correct sounds produced, level of participation)
Skill acquisition (e.g., steps of hand-washing completed, vocabulary words)
Questions teams should ask to ensure that the plan is having an impact

11. Data Collection Samples
Here are some sample data collection tools. One of your recommended take-home activities is to use these tools to practice setting up measures in your facilitator account.

12. Fidelity of Implementation
Scale of 1 (low) – 5 (high)
With what level of quality did you implement the plan?
Wk 1:
Wk 2:
Wk 3:
Wk 4:
Wk 5:
Wk 6:
Wk 7:
How many days did you implement the plan?
Wk 1:
Wk 2:
Wk 3:
Wk 4:
Wk 5:
Wk 6:
Wk 7:
One example of a way to collect data on fidelity

13. Staff Fidelity Card
Another fidelity collection example

14. Student Point Card
A way of incorporating fidelity on a CICO card

15. Combined Data Collection Sheets
Student: B.B.
Week: ­last week
Count of Disrespectful Behavior
Assignments
Fidelity (1-5) (Fridays only)
Completed
Assigned
Monday 18 1 10
Tuesday 20
Wednesday 17 11
Thursday
Absent -
Friday 19 9 3
Combined form of behavior data, academic data, and fidelity data

16. Activity:
Using a BIP from your school – determine which steps of the BIP could be assessed for fidelity and create a chart to track and monitor fidelity data
Have schools use the BIP they brought to the training to develop a plan for tracking fidelity

17. Action Plan
Goal: Establish a system for developing an evaluation and monitoring plan
Action Steps:
Develop procedures for assessing fidelity of implementation of BIP
Have teams turn to the Develop a System for Providing Ongoing Professional Development page (244) of their Tier 3 Action Plan and give them time to plan how they will address these action steps/activities and plan any other action steps/activities they wish to add.
Tier 3 Action Plan
MO SW-PBS
244

18. Individual Student Information System (ISIS-SWIS)
ISIS-SWIS is a decision system for students receiving more intensive, individualized supports for academic, social, or mental health services.
Teams using ISIS-SWIS can:
Upload and store documentation.
Define data collection measures.
Summarize data for decision making.
ISIS-SWIS is a system for storing data and team decision making

19. What is ISIS-SWIS? Individual Student Information System
ISIS-SWIS is an application within the SWIS Suite designed to coordinate and monitor individualized student support.
ISIS-SWIS allows teams to:
Set up, collect, and monitor a student’s outcome data on individualized goals based on team agreements.
Set up, collect, and monitor fidelity data about the implementation of the support plan.
Upload and store critical student plan documents needed for planning and decision making.
Summarize outcome and fidelity data for problem solving and decision making.
Benefits of incorporating ISIS-SIWS

20. Where Does ISIS-SWIS Fit?
ISIS-SWIS is a decision system for students requiring more intensive and individualized supports for academic social or mental health services.
CICO-SWIS is a decision system for targeted or group-based interventions for students needing additional support beyond the Universal or Tier 1 system.
ISIS-SWIS is the third application in the SWIS Suite. Each of the three SWIS applications (SWIS; CICO-SWIS and ISIS-SWIS) is designed to assist school personnel to make effective and efficient decisions, and together the three applications follow the multi-tiered prevention model advocated within PBIS and RtI efforts.
Tier 1: The School-Wide Information System (SWIS) is a web-based decision system designed to help school personnel to use office referral data to monitor progress of school-wide and individual student interventions.
Tier 2: CICO-SWIS is a decision system for targeted or group-based interventions for students needing additional support beyond the Universal or Tier 1 system.
Tier III: ISIS-SWIS is a decision system for students requiring more intensive and individualized supports in academics, social or mental health services.
* Note that this approach assumes that support at one level builds upon additional tiers of support. Students needing Tier III support are receiving all 3 tiers (not just individualized).
School-Wide Information System (SWIS) is a web-based decision system designed to help school personnel to use office referral data to monitor progress of school-wide and individual student interventions.

21. Advantages of ISIS-SWIS
Efficiency
Structured creation and maintenance of student files
One home for progress monitoring, goal setting, and decision making
Instantaneous access to data
Equity
Equal access to quality support plan management
Enabling of clear roles, responsibilities, and predictability
Quality
Supports compliance with federal procedures for Tier III support
Comprehensive student file for quality decision making
Documentation of progress and intervention history
Flexibility
Files and measures tailored to a student’s needs
Efficiency:
All relevant information (e.g., support plans, FBA’s, meeting minutes, grades, background information) in one place to improve meetings, support timely decision making, promote regular review of data (recommended twice weekly).
Teams can easily access and assess intervention fidelity and effectiveness to identify specific changes needed (vs. starting from scratch or making large, unnecessary changes).
Equity:
Evaluation of fidelity as well as student outcome data ensures that the student receives the right amount and type of support needed to access education.
Quality:
ISIS-SWIS allows teams to better organize documentation and easily access data in a usable format to more quickly see where and when changes to the support plan are needed.
Flexibility:
The measures, documents, data collection schedule, and reports are all set up based on the needs of the student as identified by the student’s support team.
Number of measures (fidelity and outcome)
Team member access to data
Measure details and metric types
Data collection schedule(s)
Individualized goal(s)
Wide variety of reporting options
History of plan changes and anecdotal notes

22. Efficiency
ISIS-SWIS provides structure & tools for creating and maintaining student support files.
ISIS-SWIS brings individualized student progress monitoring, goal setting, and decision making into one place.
ISIS-SWIS supports efficient data entry to keep records current.
ISIS-SWIS allows teams to easily access data for timely review and decision making.
Efficiency:
All relevant information (e.g. support plans, FBA’s, meeting minutes, grades, background information) in one place to improve meetings, support timely decision making, promote regular review of data (recommended twice weekly).
Teams can easily access and assess intervention fidelity and effectiveness to identify specific changes needed (vs. starting from scratch or making large, unnecessary changes).

23. Equity
ISIS-SWIS provides equal access to quality support for students requiring individualized interventions.
ISIS-SWIS provides staff with predictability, leading to clear roles and responsibilities.
Equity:
Evaluation of fidelity as well as student outcome data ensures that the student receives the right amount and type of support needed to access education.

24. Quality
ISIS-SWIS brings together goals/objectives from a student’s support plan (e.g., FBA, BSP, IEP) into a comprehensive student file to allow for quality decision making.
ISIS-SWIS supports compliance with federal procedures around Tier III Support (e.g., FBA, BSP, IEP).
ISIS-SWIS documents the history of student interventions and progress.
Quality:
ISIS-SWIS allows teams to better organize documentation and easily access data in a usable format to more quickly see where and when changes to the support plan are needed.

25. Flexibility
ISIS-SWIS allows progress monitoring to be individualized based solely on the team’s identification of the student’s needs.
Number of measures (fidelity and outcome)
Team member access to data
Measure details and metric types
Data collection schedule(s)
Individualized goal(s)
Wide variety of reporting options
History of plan changes and anecdotal notes
Flexibility:
The measures, documents, data collection schedule, and reports are all set up based on the needs of the student as identified by the student’s support team

26. Why was ISIS-SWIS developed?
Schools needed:
A way to efficiently enter, store, and summarize data for decision making.
A system designed for team-based planning.
A system with the flexibility to monitor progress for students with a variety of behavioral and academic needs and supports.
Developers: May, S., Talmadge, N., Todd, A. W., Horner, R. H., McGovern, S., Morris, J., Conley, K., Sampson, N., Eliason, B., & Cave, M.
Revise the developer/acknowledgements when available.

27. Why was ISIS-SWIS developed?
Overall Goal: Make schools more effective learning environments.
Strategy: Repeatedly giving people the right information, at the right time, in the right format is the single most effective way to improve decision making and valued outcomes. (Gilbert, 1978)
Behavior: Social behavior is the single most common reason students are excluded from schools.
Trainer Note:
Researchers and practitioners were working with schools and did not have a systematic way to collect, organize and analyze data for decision making. They were relying on creating their own unique spreadsheets and data tables to analyze data and evaluate effectiveness.
The goal in creating SWIS was to make schools more effective learning environments by ensuring that they could readily access the information they needed to make decisions. Creation of SWIS was about providing people the right information, at the right time, in the right format to make the best possible decisions.
The focus is on social-emotional learning & behavior because social behavior is the single most common reason that students are excluded from the learning environment (e.g., classrooms, schools).

28. Example of the student file checklist available in ISIS-SWIS

29. This is an example of the Metrics set up in ISIS-SWIS

30. Discussion Review with your team the ISIS-SWIS information.
What is your team currently using to track data?
As you continue your Tier 3 training, would ISIS-SWIS be a program your school might consider using?
How can your team use this information to improve your Tier 3 implementation?
Give teams time to discuss and share.
MO SW-PBS

31. Outcomes Today At the end of today’s session, you will be able to…
Document a plan for monitoring fidelity of Tier 3 intervention implementation.
Review Professional Development of Tier 3 systems and practices
Content learned in Day 2: Assess social validity (Day 2)
Evaluate impact of strategies (Day 2)
MO SW-PBS

32. Effective Professional Development
Key to success will be the effectiveness of the staff development you provide.
An effective intervention is one thing.
Effective implementation of an effective intervention is a very different thing.
Dean Fixsen, 2008
Providing quality professional development to your staff will be important in ensuring that your Tier 3 interventions are implemented with fidelity.
MO SW-PBS

33. Effective Professional Development
Effective professional development addresses the flaws of traditional approaches, which are often criticized for being fragmented, unproductive, inefficient, unrelated to practice, and lacking in intensity and follow-up.
Eric Clearinghouse on Teaching and Teacher Education
Can participants relate to these criticisms?
MO SW-PBS

34. Reflect on Your Own Experience
Make a t-chart for your team and label one side “Effective PD” and the other “Ineffective PD”
What are the characteristics of effective professional development you have received?
What are the characteristics of ineffective professional development you have received?
List these characteristics on your team’s t-chart
Once teams have completed their t-charts, have them share with the group.
Are there common characteristics that a majority of groups mentioned for effective and ineffective professional development?
Each team’s challenge is to provide effective professional development to their school staff on the systems, data, and practices for Tier 3 intervention.
MO SW-PBS

35. Effective Professional Development
Research-based
Consistent, ongoing
Convenient
Relevant
Differentiated
Nobori, 2011
Research-based – let research guide your strategies (SW-PBS is research-based, including the Tier 3 intervention that you will be implementing – FBA/BIP)
Consistent – create a long-term plan for PD and be consistent in the implementation across multiple years (The professional development you provide your staff will not be one-time at the beginning of the year before school starts. Staff development needs to be on-going, including new learning as well as “booster” sessions for review. Your staff will need to be trained so they can develop the FBA/BIP expertise needed to be contributing members to Action Teams. Additionally, staff will need to receive training in order to implement BIP strategies and collect data.
Convenient – schedule at a time that increases the likelihood of attendance (When will this be for your school? Your team will need to work closely with your administrator to ensure that professional development is a priority)
Relevant – focus PD on enhancing your school’s instructional model (Teachers will see a direct benefit in the improved behavior and positive relationships of the students participating in a Tier 3 intervention)
Differentiated – provide choices when possible. i.e. an instructional approach and a technology approach (Does all professional development in your school need to be face-to-face, large group? Is using technology such as podcasts and webinars an option? What are other possibilities?)
MO SW-PBS

36. Quality Professional Development
Other characteristics:
Includes training, practice, feedback and coaching or other follow-up procedures/support
Provides opportunities for teachers to interact with peers
Ismat, 1996
This is a partial list from the Eric Clearinghouse on Teaching and Teacher Education.
Did the teams identify a lot of the characteristics that were on these two slides?
You will see the impact of including training, practice, feedback, and coaching on the next slide.
MO SW-PBS

37. How Did You Do?
Compare the "Effective PD" column of the t-chart your team created earlier with the characteristics described by Norobi and Ismat. Are there similarities?
Give teams time to compare and discuss.
MO SW-PBS

38. Demonstrate Knowledge Demonstrate New Skill in Training
 OUTCOMES
(% of Participants who: Demonstrate Knowledge, Demonstrate New Skills in a Training Setting, and Use New Skills in the Classroom)
Training Components
Demonstrate Knowledge
Demonstrate New Skill in Training
Use New Skills in Classroom
Theory and Discussion
10% 5% 0%
Plus Demonstration in Training
30%
20%
Plus Practice and Feedback in Training
60%
Plus Coaching in the Classroom
95%
After an analysis of more than 200 studies of in-service training methods, Joyce and Showers (1980) found that when the goal of training is related to the integration and transfer of complex ideas or requires significant modifications to teaching method, only the combination of all five components--theory, demonstration, practice, feedback, and coaching--was consistent with the transfer of training to the classroom (Joyce & Showers, 2002).
After a few decades of research on training teachers, Joyce & Showers (2002) began to think of training and coaching as one continuous set of operations designed to produce actual changes in the classroom behavior of teachers. One without the other is insufficient.
Now is the time for your team to start thinking how you can provide Tier 3 training to your staff that includes the components identified by Joyce and Showers.
187
Joyce and Showers, 2002

39. Discussion
Discuss the findings of Joyce and Showers and how this information impacts the type of professional development you will need to provide your staff.
Give teams time to compare and discuss.
Ask: What role does your school counselor play in your Tier 3 PD system? According to the recent document created by a partnership between MSCA, DESE and SW-PBS entitled: “The Role of the Professional School Counselor in Implementing SW-PBS” (October 2013), The Professional School Counselor plays an integral role in all three SW-PBS Tier Teams as a resource by contributing professional knowledge to the development, implementation, and evaluation of the Tier procedures and interventions; promoting greater continuity between tiers; and being trained at each tier to fully understand and contribute to the success of SW-PBS. The counselor can share information and expertise in areas that align with the unique role and training of the Professional School Counselor, such as the science of behavior; confidentiality guidelines, regulations and rationale; and training in how to conduct observations.
MO SW-PBS
187

40. Finding Time
Can staff meetings have an instructional focus instead of a managerial focus?
Can portions of grade level/departmental planning times be used for staff development?
Can classes/activities be covered by other available school personnel for teachers to attend staff development?
Can technology be utilized to allow for flexible training times?
Other ideas?
Each school is unique in schedules, budgetary restrictions for substitutes, coverage availability from other staff, and contractual/negotiated agreement restrictions.
All schools also have competing initiatives. Teams can inventory those initiatives – are all of them current and necessary? (Many times schools continue to do something because “it’s always been that way” even though it is no longer effective or needed.) Do some of the initiatives have overlapping purposes and can be streamlined?
Have teams share ideas that have worked for them in getting more time for staff development.
Teams will need to work closely with their administrators in order to develop a professional development calendar.
MO SW-PBS

41. Discussion
What options are available in your school for scheduling Tier 3 professional development opportunities?
How far in advance is the professional development calendar established?
Do you need to start planning now to get additional time on the calendar for next year?
Give teams time to discuss.
MO SW-PBS

42. Updating Staff
Develop standard procedures for providing updates to administrator, team, staff and others involved with student support
What will be communicated?
Who will communicate this information?
How will it be communicated?
When will it be communicated?
These four questions will assist your team in developing your communication system; if you can answer these questions, your system for updating staff is in place.
MO SW-PBS

43. Discussion
Does your Tier 3 team have an effective system already established for updating staff? If not:
How do your Tier 1 and Tier 2 teams provide regular staff updates?
If that system is effective, can your Tier 3 team incorporate the same system?
Give teams time to discuss. Ask them to share effective systems already in place.
MO SW-PBS

44. Action Plan
Goal: Develop a system for providing ongoing professional development
Action Steps:
Develop a system for regularly updating staff
Create a calendar of ongoing professional development regarding Tier 3
Have teams turn to the Develop a System for Providing Ongoing Professional Development page (244) of their Tier 3 Action Plan and give them time to plan how they will address these action steps/activities and plan any other action steps/activities they wish to add.
Tier 3 Action Plan
MO SW-PBS
244

45. Tier 3 Staff Handbook
Your Tier 3 staff handbook will serve as a reference for staff and document your Tier 3 data, systems, and practices
Information in your Tier 3 Staff Handbook will provide the basis for your Tier 3 professional development
As you go through this year, you will be developing your Tier 3 staff handbook; this handbook will include information that will be content for the professional development you provide your staff.
The handout provides guidance for what you will include in your handbook as you develop your systems, data, and practices.
Tier 3 Staff Handbook Organizer
MO SW-PBS
200

46. Discussion Review the Tier 3 Staff Handbook Organizer.
What do you already have available that can be added to your Tier 3 Staff Handbook?
As you continue your Tier 3 training, who will be responsible for adding/updating your Tier 3 Staff Handbook?
How will staff access the handbook? Will you provide hard copies? Electronic copies?
Give teams time to discuss and share.
Tier 3 Staff Handbook Organizer
MO SW-PBS
200

47. Action Plan
Goal: Develop a system for providing ongoing professional development
Action Steps:
Develop- a Tier 3 Staff Handbook
Have teams turn to the Develop a System for Providing Ongoing Professional Development page (244) of their Tier 3 Action Plan and give them time to plan how they will address these action steps/activities and plan any other action steps/activities they wish to add.
Tier 3 Action Plan
MO SW-PBS

48. Next Session: T3A Session 4 Using Environmental Interventions
March 30, 2016
Edplus Building Annex
Things to Bring:
Laptop, Tier 3 workbook, Action plan, Items needing to be worked on
MO SW-PBS

49. Contact Information: Dr. Deb Childs 636/