1. The Systems Coach Manual Effective Behavioral & Instructional Support Systems (EBISS) Erin A. Chaparro, Ph.D. Kathleen Ryan Jackson

2. Welcome Erin Chaparro, Ph.D. UO EBISS Co-Principal Investigator echaparr@uoregon.edu Kathleen Ryan Jackson UO EBISS Coordinator kmj@uoregon.edu http://ctl.uoregon.edu/pd

3. Presentation Objectives 1.Describe project goals 2.Define critical components of EBISS and Systems Coaching 3.Introduce Systems Coaching tools used in the Effective Behavioral and Instructional Systems Support (EBISS) initiative

4. EBISS GOALS 1.Teach district leaders how to implement PBIS and the school-wide reading model system-wide Data – Systems – Practices - Outcomes 2. Increase efficient use of district and building resources for response to instruction and intervention (RTI) Recourse Allocation Based on Need 3. Develop capacity for high-quality, sustainable implementation of evidence based practices Distributed Leadership

5. 26 School Districts: 2006-2012 Research supported by the Oregon Department of Education, ED Grant Number H323A060007 to the University of Oregon

6. Critical Components of EBISS 1.Representative District and Building Leadership Teams 2.The EBISS Teaming Framework Evidence-based practices and Outcomes 3.The District Systems Support Plan Technical Adequacy 4.Systems Coach Manual Implementation Tools

7. Critical Component #1 District and Building Leadership Teams

8. EBISS: Whole System Reform District and Building Leadership – Crystal clear focus, direction and target Build Capacity from Day One – Same focused goals, same language – Distributive Leadership, leaders developing leaders Developing a Strong System of Schools – Not one or two strong schools Harris (2008) Fullan (2010) Spillane (2012)

9. Implementation Team A group that knows the evidence based practices very well (formal and craft knowledge) A group that knows implementation very well (formal and craft knowledge) A group that knows improvement cycles to make intervention and implementation methods more effective and efficient over time

10. Implementation Team Minimum of three people (four or five preferred) to promote effective, efficient, and sustainable implementation, organization change, and system transformation work Tolerate turnover; teams are sustainable even when the players come and go

11. EBISS Critical Component #2 EBISS Teaming Framework & Evidence-based Practices

12. EBISS Teaming Framework

13. EBISS Teaming Framework: District Leadership

14. EBISS Teaming Framework: Building Leadership

15. Evidence-based Practices Outcomes School Level Data Summary Oregon Assessment of Knowledge and Skills

18. Literacy Benchmarking Third Grade Across 4 years First Grade Across 4 years

19. Oregon Assessment of Knowledge and Skills Changes Across 4 Years 3 rd Grade07-0808-0909-1010-11 Met49.352.951.549.5 Exceeded35.530.933.933.6 4th Grade07-0808-0909-1010-11 Met45.546.445.846.3 Exceeded39.139.039.738.1 5th Grade07-0808-0909-1010-11 Met50.151.952.953.2 Exceeded25.425.325.424.8

20. EBISS Critical Component #3 District Systems Support Plan (DSSP)

21. LEADERSHIP & COMMITMENT ACTION PLANNING with SCHOOLS COORDINATION & COACHING PROFESSIOANL DEVELOPMENT & TRAINING CAPACITY ON-GOING ASSESMENT & EVALUATION FUNDING VISIBILITY & POLITICAL STAKEHOLDER SUPPORT District Systems Support Plan (DSSP) Critical Variables for Program-Wide Infrastructure Data for Action Planning and On- Going Evaluation

23. District Systems Support Plan (DSSP) Technical Adequacy

24. DDSP Summary 7 Features 23 Items in total Each feature has in between 2 – 8 items Likert-Scaled o 3 = Completely in place o 2 = Mostly in place o 1 = Partially in place o 0 = Not in plan

25. DSSP: Internal Reliability 23 items in total; Cronbach’s Alpha =.947 For each Feature: Funding =.822 Visibility =.736 Assessment =.795 Training =.759 Coordination =.826 Action Planning =.791 Leadership =.839

26. DSSP: Confirmatory Factor Analysis

27. DSSP: Construct Validity 1) The model chi- square The chi-square value was 425.482 (df = 209) and it was statistically significant indicating poor fit (p <.001). It is known to be problematic because it is sensitive to the sample size and the size of the correlations. 2) The root mean square error of approximation (RMSEA) Typically, the RMSEA values less than.10 are considered reasonable fit. In our model, the RMSEA index was.09 (90% confidence interval:.80 -.10) indicating reasonable fit. 3) The comparative fit index (CFI) The CFI assesses the relative improvement in fit of the researchers’ model compared with the null model which assumes no covariance among the observed variables. The CFI index larger than.90 indicates reasonably good fit of the researcher’s model. Our model demonstrated CFI =.87. 4) The standardized root mean square residual (SRMR) SRMR is based on the overall difference between the observed and predicted correlations in the data. SRMR values less than.10 are generally considered favorable. The SRMR index for our model was.06 implying a good fit

28. Convergent Validity DISTRICT SYSTEMS SUPPORT PLAN

29. † p <.10, * p <.05, ** p <.01, *** p <.001

30. DSSP: Summary DSSP has promise as a team self-assessment fidelity measure Correlations between the DSSP constructs and: – ‘district leadership and commitment’ construct had positive correlations that are at least marginally significant across all grades – the percentage of students in special education programs were negative implying that the districts with higher DSSP values tend to have fewer students identified as eligible for special education. – positive correlations were more prominent in higher grades such as grades 8 and 11 for all the DSSP constructs (EBISS targets more elementary practices).

31. EBISS Critical Feature #4 The Systems Coach Manual Oregon Coaches Task Force, 2012

32. The Systems Coach Manual Implementation Tools 1.EBISS Framework Membership & Purpose 2.Coaches Self-Assessment 3.DSSP Scoring Guide 4.Implementation Workbook

33. Coaching a District System Identifies: Coaching Traits Strong interpersonal and intrapersonal skills Identifies: Coaching Knowledge and Skills Develop shared commitment through goal setting and action planning Identifies: Content Knowledge and Skills Multi-tiered model of service delivery: RTI Framework

34. EBISS Teaming Framework Recommended Membership and Purpose District Leadership Team Literacy and Behavior Team/Coaches School Leadership Team Grade, Content, Individual Teams

35. EBISS Coaches Self Assessment Across Each Construct of the DSSP

36. DSSP Scoring Guide Identifying Best Practice

37. Implementation Workbook BLUEPRINT FOR RESPONSE TO INSTRUCTION FOR PBIS AND ACADEMIC SUPPORT TRAINING AND PROFESSIONAL DEVELOPMENT Adapted for EBISS State/District Implementation Workbook National Technical Assistance Center on Positive Behavioral Interventions and Supports Susan Barrett, Timothy J. Lewis, George Sugai, Rob H. Horner

38. Guiding Questions Professional Development &Technical Assistance Facilitated Exploration Activities Outcomes Exploration

39. Erin Chaparro, Ph.D. UO EBISS Co-Principal Investigator echaparr@uoregon.edu Kathleen Ryan Jackson UO EBISS Coordinator kmj@uoregon.edu QUESTIONS

40. References & Resources Chaparro, E. A., Park, H., Baker, S. K., & Ryan-Jackson, K. M. (2011). District System Support Plan: A District Level Self-Assessment Tool (Report No. 1102). Eugene, OR, Center on Teaching and Learning, University of Oregon. Chaparro, E. A., Ryan Jackson, K.M., Baker, S. K., & Smolkowski, K. (2012). Effective Behavioral and Instructional Support Systems: An Integrated Approach to Behavior and Academic Support at the District Level. School Based Mental Health Interventions 5, 161-176. Chaparro, E. A., Smolkowski, K., Baker, S. K., Hanson, N. & Ryan-Jackson, K. M. (2011). Closing the Implementation Gap with Increased Collaboration on Behavior and Literacy Evidence-Based Practices. Manuscript submitted for publication. Effective Behavioral and Instructional Support Systems model (EBISS; http://ctl.uoregon.edu/pd) http://ctl.uoregon.edu/pd Oregon Coaches Task Force. (2011). K-12 Systems Coach Manual. Center on Teaching and Learning, University of Oregon, Eugene, OR. http://ctl.uoregon.edu/pdhttp://ctl.uoregon.edu/pd Scaling Up EBISS BlogSpot: http://ebissscalingup.blogspot.comhttp://ebissscalingup.blogspot.com

41. References & Resources (cont.) Fullan, M. (Ed.). (2009). Motion leadership: The skinny on becoming change savvy. Corwin Press. Fixsen, D. L., Blase, K. A., Timbers, G. D., & Wolf, M. M. (2001). In search of program implementation: 792 replications of the Teaching-Family Model. In G. A. Bernfeld, D. P. Farrington & A. W. Leschied (Eds.), Offender rehabilitation in practice: Implementing and evaluating effective programs (pp. 149-166). London: Wiley Goodwin, L.D., (2002). The Meaning of validity. Journal of Pediatric Gastroenterology & Nutrition, 1, 6-7. Harris, A. (2008). Distributed leadership: According to the evidence. Journal of Educational Administration, 46(2), 172-188. Honig, M. I., Copland, M. A., Rainey, L., Lorton, J. A., & Newton, M. (2010). Central office transformation for district-wide teaching and learning improvement. Retrieved from The Wallace Foundation website: http://www.wallacefoundation.org/knowledge-center/school-leadership/district-policy- and-practice/Documents/Central-Office-Transformation-District-Wide-Teaching-and-Learning.pdfhttp://www.wallacefoundation.org/knowledge-center/school-leadership/district-policy- and-practice/Documents/Central-Office-Transformation-District-Wide-Teaching-and-Learning.pdf Johnson, E., Mellard, D.F., Fuchs, D., & McKnight, M.A. (2006). Responsiveness to intervention (RTI): How to do it. Lawrence, KS: National Research Center on Learning Disabilities http://www.nrcld.org/rti_manual/pages/RTIManualSection4.pdf http://www.nrcld.org/rti_manual/pages/RTIManualSection4.pdf

42. Resources & Resources (cont.) Johnson, P. (2010). Leading for learning. Leadership practices of effective boards. Spectrum, 28, 27-42. Klein, R.B. (2011). Principles and practice of structural equation modeling. Gilford Press: New York. Mowbray, C.T., Holter, M.C, Teague, G.B. and Bybee, D. (2003). Fidelity criteria: Development, measurement, and validation. American Journal of Evaluation, 24, 3, 315-340. doi: 10.1177/109821400302400303 Robinson, V. M. J., Lloyd, C. A., & Rowe, K. J. (2008). The impact of leadership on student outcomes: an analysis of the differential effects of leadership types. Educational Administration Quarterly, 44, 635–674. doi: 10.1177/0013161x08321509 Samuels, C.A. (2013). Job roles shifting for Districts’ Central Offices. Retrieved from Education Week Website http://sisep.fpg.unc.edu/ http://sisep.fpg.unc.edu/ Spillane, J. P. (2012). Distributed leadership (Vol. 4). Jossey-Bass. State Implementation and Scaling Up of Evidence Based Practices (SISEP: http://sisep.fpg.unc.edu/)http://sisep.fpg.unc.edu/ Thorndike, R.M., Thorndilke-Christ, T.M. (2010). Measurement and evaluation in psychology and education. San Francisco: Pearson