1. Steve Goodman, MiBLSi, Michigan Rob Horner, University of Oregon
Building and Measuring District Capacity to Implement MTSS at all Three Tiers
Steve Goodman, MiBLSi, Michigan
Rob Horner, University of Oregon
OSEP conference banner on top
PBIS TA-Center Logo on bottom
How to get help with audio issues: Antony at
How to participate in Polls and Comments: “Use the Box on the Right"

2. Goals Define District Capacity
Define when, why and how to build District Capacity
For LEAs and for SEAs
Propose a measure of District Capacity
Provide examples
Picture of teacher working with a group of six students

3. Define: District Capacity
District capacity is defined as the systems, activities, and resources necessary to adopt, implement, and sustain a practice (Ward, et al. 2015)

4. Content Validity (Tier 1 .95; Tier II .93; Tier III .91)
Usability (12 of 14 > 80%) (15 min per Tier)
Inter-rater Agreement (.95; .96; .89)
Test-retest reliability (.98; .99; .99)
Factor Analysis
Available from OSEP TA-Center
No Cost
Assessors Training PowerPoint
and
Assessors Training Video at

5. Sub-scale report

6. Poll #1

9. Personnel Selection, Orientation, Evaluation
Annual Evaluation
Annual evaluations include assessment of the extent to which a faculty/staff member has delivered multi-tiered academic, behavioral and emotional supports.
Recruit and Hire
“Preference will be given to applicants with demonstrated experience and expertise in implementation of multi-tiered systems of academic and behavior support.”
Start of Year Orientation
Orientation establishes (a) importance of school-wide social culture, (b) commitment to equity and MTSS, (c) personal role of each person in the school community

10. Professional Development
Training and Professional Development
District trainers regularly provide training in core features of effective academic and behavior supports. Training events are integrated across year.
Coaching in Schools
District coaches support use of new skills in typical school contexts

11. Data Systems
Providing the right data in the right form at the right time
Providing the training in how to use data for problem solving.

12. Ward, C. , St. Martin, K. , Horner, R. , Duda, M. , Ingram-West, K
Ward, C., St. Martin, K., Horner, R., Duda, M., Ingram-West, K., Tedesco, M., Putnam, D., Buenrostro, M., & Chaparro, E. (2015). District Capacity Assessment. University of North Carolina at Chapel Hill

13. District Capacity Assessment
Content
10 subscales- 40 Items
Commitment and Leadership
Systems Alignment
Action Planning
Performance Feedback
Selection
Training
Coaching
Decision-support System
Facilitative Administration
Systems Intervention

14. Scoring Form on SISEP.org
Data Entry
Scoring Rubric
Data Source
Item
Team Vote
Majority score is what is recorded
“Are we ready”… “your score”
Consensus means that voters in the minority can live with and support the majority decision on an item.

16. Action Planning
Step 1: For any item listed below a “2” consider actions that may be completed within the next 3 months.
Step 2: Define the action, "who" is responsible, when it will be accomplished, and the team/meeting when updates on the action will be reviewed.
Step 3: Team should prioritize the areas or items that are most critical to improve– critical defined as most likely to improve fidelity, sustainability and student outcomes.

18. Poll #2

19. Implementing PBIS in Springfield School District
Nicole Nakayama, Brian Megert
Springfield School District #19, Oregon

20. PBIS Implementation in Springfield School District, Oregon
22 schools and 11,000 students
District Capacity
Long history of implementing multi-tiered literacy practices
All Elem
All Middle & High Schools
5 Model Demo Schools
Tier I
Tier II
Tier III
All Elem
5 Model Demo Schools
Graphic indicating gradual implementation of PBIS in the district. Starting with model demo sites, then scaling to other schools in Springfield School District
5 Model Demo Schools

21. DCA Total Score- Springfield PS 2013-14 (PBIS)

22. DCA sub-scale scores

23. DCA Total Scores- Springfield PS 2013-14, and 2014-15 (PBIS)

24. DCA Total Score- Springfield PS 2013-14; 2014-15; 2015-16 (PBIS)

25. Building District Capacity
DCA Total Score
Histogram showing Total Score for District Capacity Assessment for Springfield School District for ; and
Data document improved district capacity in PBIS implementation over the past three years, and that capacity for early literacy implementation is lower, emerging, but in need of more development.

26. Tier I PBIS Implementation: Model Demonstrations 2013-14, 2014-15, 2015-16

27. Tier II PBIS Implementation: Model Demonstration
TFI Tier II Total Score
Tier II TFI data for five schools
Four of five schools met Tier II implementation criterion after one year of TA

28. Tier III PBIS Implementation Model Demonstrations

29. Scaling Up PBIS within SPS 2014-15 and 2015-16
Histograms of TFI total scores for Tier I, Tier I and Tier III for 18 NON-Demo schools in Springfield school district.
These data document that the district was effective at scaling up PBIS

30. Suspensions per 100 students All Elementary Schools

31. Building District Capacity
District Leadership Team
Authority, Membership, Meeting Schedule/format
District Data System
What to collect, when, by whom, how used
Skill Development
Use demonstration schools to build district technical capacity
District Trainers (PBIS [Tier I, Tier II, Tier III]
Coaching capacity
School psychologists, counselors, social workers

32. District Capacity Assessment: Examples from State and Local Levels
Tuesday November 8, 2016
2:20PM – 3:00PM
Steve Goodman
MiBLSi

33. Using Data for Action Planning
High
Just do it!
Maybe a quick win
Consider cost vs. benefit
Impact
Continue performing root cause analysis
Maybe a quick win
Consider cost vs. benefit
Low
Low
High
Ease of Implementation

34. Prioritizing suggestions for linking DCA to action plans
Focus on high impact items
Focus on items that are prerequisite for other items
Focus on items that “almost there” to keep momentum moving forward- “low hanging fruit”
Focus on items that are already connected with others initiatives- “in the works”

35. Focus on high impact items
DCA Item #
DCA Item 1
There is a District Implementation Team (DIT) to support implementation of Effective Innovations (EI) 2
DIT includes an individual with executive leadership authority 7
Funds are available to support the implementation of the EI 8
DIT has an implementation plan for the EI 15
DIT has a process for using data for decision making

36. Focus on items that are prerequisite for other items
1. There is a District Implementation Team (DIT) to support implementation of Effective Innovations (EI)
is a prerequisite for
4. DIT uses an effective team meeting process
8. DIT has an implementation plan for the EI
is a prerequisite for
9. DIT continuously improves the use of the Implementation plans

37. District level use of DCA data

38. Port Huron Area School District
Total Score
Organization
Leadership
Competency
Restructured team approach Fall 2015
Team membership
Director of Student Services and Behavior Supports
Supervisor of Behavioral Support and Professional Development
Five District behavior coaches
Meeting schedule
Monthly meetings with district senior administration
Weekly meetings with implementation team
Completed by Executive Director of Instruction, Executive Director of Student and Staff Services, Director of Federal and State Programs and Grants, Director of Instruction, Director of Student Services and Behavior Supports, Supervisor for Behavior Supports

39. Port Huron Area School District Implementation/Leadership Team Meeting Agenda
our plan to address those items scored less than a 2.  Our next step is to take those to an action plan. 

40. Port Huron Area School District:
District Capacity and PBIS School Tier 1 Fidelity
District
Schools in District
Fidelity criteria
6 Schools Represented with 2 years of data. 16 schools within district
10/16 schools are collecting fidelity data in and 8/16 have achieved fidelity criteria

41. Swartz Creek Community Schools
In June the DCA was completed by the Superintendent, Asst. Superintendent of Curriculum and Instruction, the Middle School Principal, the High School Principal, an Elementary Principal, and the Director of Student Services (Special Ed.). Typically, as the District MTSS Coordinator, I would also be present as part of the team.  I have facilitated the process as well. 
Completed by the Superintendent, Asst. Superintendent of Curriculum and Instruction, Middle School Principal, High School Principal, Elementary Principal, Director of Student Services (Special Ed.) and MTSS Coordinator

42. Swartz Creek Community Schools Action Plan
In June the DCA was completed by the Superintendent, Asst. Superintendent of Curriculum and Instruction, the Middle School Principal, the High School Principal, an Elementary Principal, and the Director of Student Services (Special Ed.). Typically, as the District MTSS Coordinator, I would also be present as part of the team.  I have facilitated the process as well. 

43. Swartz Creek Community Schools: DCA Planning Example
DCA Scoring
Action Plan

44. Swartz Creek Community Schools

45. Poll #3

46. State level use of DCA from local districts to support capacity development

47. Project-wide DCA: Criterion and Growth

48. Project-wide DCA: Criterion and Growth
Met Criterion
Met Criterion and Made 10% Growth

49. DCA Total Score Median Across Time by Cohort

50. Project plan to support implementation of local capacity

51. Project plan to support implementation of local capacity

52. Port Huron Area School District
Total Score
Organization
Leadership
Competency

53. Project data-base with section for notes
5/29/15
9/16/15
Assistant Superintendent for Instruction regularly participates in meetings
Mary T. is District PBIS coordinator

54. Poll #4

55. Lessons Learned
We need to be clear on why we conduct the DCA (help to develop efficient, effective and sustainable systems)
To be meaningful, we need the right people participating in conducting the DCA (e.g., administrators, content expertise)
The discussion during the process is valuable for learning and creating vision for work
It is critical to link DCA results to action plan

56. Questions/ Comments