Assessing Implementation at the Classroom level Lori Cameron: WI RtI Center/PBIS Network John Jagemann: Milwaukee Public Schools Megan Mills Koehler: Sunnyside Elementary School, Pulaski

Objectives Provide resources to assess implementation of PBIS at the classroom level Share examples of how this was done at both district and school levels

High Student Engagement = High Achievement Factors that most impact student engagement Classroom management is also strongly associated with student academic engagement. In 1994, Wang and others analyzed over 11,000 statistical findings to determine those factors that correlated the highest with student academic achievement, and rank ordered 28 variables. Classroom management was at the top of the list. Despite this statistic, only a third of teaching colleges teach classroom management. Clearly there is a need for staff development in the area of classroom management. But what constitutes sound professional development? Citation: Lori Newcomer and Jeff Sprague High Student Engagement = High Achievement

What Actions Produce Implementation? Presentation of Theory Understanding (Explain main concepts) Skill Attainment (Can demonstrate) Use with Fidelity (Uses the skill effectively and consistently) Presentation of Theory + Lecture, cooperative learning 85% 15% 5%-10% Modeling + Modeling, video, simulation 85% 18% 5%-10% W have known for nearly 30 years, from the work of Bruce Joyce and Beverly Showers, Jim Knight, and others, that the “Train and Hope” model of professional development doesn’t work very well. Training certainly provides teachers with a significant understanding of new practices. After a training, most are able to explain what the new practice is. However, once they are “trained,” only a small percentage of teachers are able to demonstrate the new skill, even though they can explain it, and even fewer use the new practice effectively and consistently after experiencing training. Modeling the new practice for teachers in addition to the intital training slightly increases the ability to demonstrate it, but does not increase the consistent and effective use of the new practice. Actually practicing the new skill during training significantly increases teachers’ ability to demonstrate it, but only slightly increases their use ot it in their classrooms. It is only when coaching is added to the professional development scope and sequence, as a follow-up to training, modeling, and practice and feedback, that teachers consistently and effectively use the new classroom management practices in their classrooms. Practice & Feedback + Practice during learning session 85% 80% 10%-15%

Classroom Management On-Line Resource http://www.wisconsinpbisnetwork.org

Assessment SAS: Classroom BOQ: Classroom Knowledge Surveys Self Reflection Check Lists

PBIS Assessments BOQ SAS Classroom Current Status

Knowledge Pre and Post Survey

Knowledge Survey 8 multiple choice questions Takes about 10 minutes to complete Provides a copy of your answers for the pre test Provides the answers for the post test

Self Reflection Checklists

What Actions Produce Implementation? Presentation of Theory Understanding (Explain main concepts) Skill Attainment (Can demonstrate) Use with Fidelity (Uses the skill effectively and consistently) Presentation of Theory + Lecture, cooperative learning 85% 15% 5%-10% Modeling + Modeling, video, simulation 85% 18% 5%-10% W have known for nearly 30 years, from the work of Bruce Joyce and Beverly Showers, Jim Knight, and others, that the “Train and Hope” model of professional development doesn’t work very well. Training certainly provides teachers with a significant understanding of new practices. After a training, most are able to explain what the new practice is. However, once they are “trained,” only a small percentage of teachers are able to demonstrate the new skill, even though they can explain it, and even fewer use the new practice effectively and consistently after experiencing training. Modeling the new practice for teachers in addition to the intital training slightly increases the ability to demonstrate it, but does not increase the consistent and effective use of the new practice. Actually practicing the new skill during training significantly increases teachers’ ability to demonstrate it, but only slightly increases their use ot it in their classrooms. It is only when coaching is added to the professional development scope and sequence, as a follow-up to training, modeling, and practice and feedback, that teachers consistently and effectively use the new classroom management practices in their classrooms. Practice & Feedback + Practice during learning session 85% 80% 10%-15% Coaching Where work occurs 90% 90% 80%-90%

Walk Throughs

Rationale Provides a non-evaluative, objective snap shot of implementation Versatile in use Can be used for pre and post Discussion starter

Rationale Allows teachers to see other classrooms Identifies key features of a strategy Can be multi focused and single strategy fouced Supplements TFI (Tiered Fidelity Checklist)

Versatile Uses Conducted by coach or teams of teachers Whole school results can be aggregated Individual teacher results can be shared privately Do whole school, or a random sampling Announce dates of walk through or do unannounced

Examples Walk throughs

Walk through Examples Missouri New Jersey (May Institute) 15 page comprehensive tool Page 7 has a tally for ratio of positives to correctives Includes a 20 minute observation Rules and Procedures Detailed observation form for ratio of positives and correctives On task and off task behavior

Walk through Examples MPS CLR Learning Environment Survey MPS Created by Sharroky Hollie’s to assess Cultural Responsive classroom Looks at 5 basic skills 5 classrooms per school Aggregates scores Include brief student interview

Wisdom from the Field John Jagemann Megan Mills Koehler

PBIS Leadership Conference Jon Jagemann MPS RtI PBIS District Supervisor August 17th, 2016 ©2015 Milwaukee Public Schools

Classroom Best Practices Teach Expectations/Procedures Re-direct/Set Consequences Set Expectations/Procedures Acknowledge Build Relationships 1 min ©2015 Milwaukee Public Schools

Culturally Responsive Classroom Best Practices Teach Expectations/Procedures Re-direct/Set Consequences Set Expectations/Procedures Acknowledge Build Relationships 1 min ©2015 Milwaukee Public Schools

Culturally Responsive Classroom Best Practices Set Expectations, Rules, and Procedures Teach Expectations, Rules, and Procedures Re-direct/Set Consequences Build Relationships Learn about students’ cultures Address students by name Teach students about you and your culture Allow opportunities for students to share their cultures Acknowledge 1 min ©2015 Milwaukee Public Schools

Culturally Responsive Classroom Best Practices Set Expectations, Rules, and Procedures Teach Expectations, Rules, and Procedures Re-direct/Set Consequences Understand expectations and norms of various cultures Allow for student voice in creation Build Relationships Acknowledge 1 min ©2015 Milwaukee Public Schools

Culturally Responsive Classroom Best Practices Set Expectations, Rules, and Procedures Teach Expectations, Rules, and Procedures Re-direct/Set Consequences Build Relationships Teach behaviors in multiple ways for all learning styles Teaching behaviors should be student focused Have posted throughout room for all students to use as a guide Acknowledge 1 min ©2015 Milwaukee Public Schools

Culturally Responsive Classroom Best Practices Set Expectations, Rules, and Procedures Teach Expectations, Rules, and Procedures Re-direct/Set Consequences Build Relationships Respect student individuality and cultures through re-direction and consequences Allow for student to maintain dignity and individuality through process Acknowledge 1 min ©2015 Milwaukee Public Schools

Culturally Responsive Classroom Best Practices Set Expectations, Rules, and Procedures Teach Expectations, Rules, and Procedures Re-direct/Set Consequences Build Relationships Acknowledge individuality of student Ensure all students receive 5:1 positives to corrective interactions with adults Acknowledge 1 min ©2015 Milwaukee Public Schools

PD Modules Session 1: Building a Climate for Learning Session 2: Building a Relationship with all students Session 3: Re-direction of Students

Feedback Tool

Connection to Danielson Teacher sets (and posts) expectations and procedures for students in all settings. (Danielson 2A and 2B) 2. Teacher teaches all expectations and procedures including pre-correcting before all transitions. (Danielson 2C and 2D)

Connection to Danielson 3. Teacher re-directs and sets consequences to students who are not meeting the expectations in a manner that de-escalates the situation and allows the student to meet the expectation going forward. (Danielson 2C and 2D) 4. Students are positively reinforced with positive language when good choices are made and are given corrective feedback when needed. (Danielson 2A and 3D)

Connection to Danielson 5. Teacher builds relationships with all students and shows interest in all students. Positive climate is created in classroom. (Danielson 1B and 2A)

Feedback Tool Can be used: Self-Reflection Tool Peer Visits School Support Teacher (SST) Administration (non-evaluative) District PBIS Coach

PBIS Leadership Conference MPS Board of School Directors MPS Senior Team Mark Sain, President, District 1 Larry Miller, Vice President, District 5 Wendell J. Harris, Sr., District 2 Michael Bonds, Ph.D., District 3 Annie Woodward, District 4 Tatiana Joseph, Ph.D., District 6 Claire Zautke, District 7 Carol Voss, District 8 Terrence Falk, At-Large Darienne B. Driver, Ed.D., Superintendent Gina Spang, P.E., Chief of Staff Tonya Adair, Chief Innovation & Information Officer Daniel Chanen, J.D., Chief Human Resources Officer Ruth Maegli, Chief Academic Officer Gerald Pace, J.D., Chief Financial Officer Keith Posley, Ed.D., Chief School Administration Officer Wendell Willis, Chief Operations Officer Sue Saller, Manager, Superintendent’s Initiatives Ashley Lee, Special Assistant to the Superintendent

Wisdom from the Field John Jagemann Megan Mills Koehler

Demographics Sunnyside Elementary is 1 of 5 elementary schools within the Pulaski Community School District We currently have an enrollment of approximately 495 students 4K-5 We service a primarily rural community 96.7% of students are Caucasian Students with a disability 13.5% 11.4% of students are considered economically disadvantaged

PBIS Journey We have been implementing TIER I for 4 years and began TIER II this past 2015-2016 school year Our elementary school houses the severe EBD program for the entire district We utilize a co-teaching model in our special education program K-12 This past school year we had 22 new staff members including a new principal.

Providing Professional Development To Support PBIS in the Classroom Creating Buy In What’s in it for me? How’s that working for you? Why Professional Development? Research on impact of effective classroom management and student engagement Provide teachers with what they need to be effective in the classroom Resources/Time Schedule it at the beginning of the year Have a focus 20-25 minutes WI PBIS Network Classroom Modules CHAMPS Social Thinking

Sunnyside Walkthrough Process Classroom Matrix Completed two times a year PBIS team members conduct the walkthroughs with look-fors using rubric Feedback given to teachers with ideas for improvement Student Interview Conducted two times a year 3 students from each class randomly interviewed using Student Interview Rubric/Interview PBIS team members conduct interviews

Classroom Matrix Rubric PBIS Classroom Matrix Implementation Criteria # Criteria Not In Place Partially In Place In Place 1 School-wide expectations Be Safe, Be Respectful, Be Responsible are displayed. 2 Procedures/Expectations are identified for at least 4 activities within the classroom. These should include arrival, transitions, cooperative learning groups, and whole group instruction. Specialists should at minimum address 2 areas. 3 Procedures/Expectations should be positively stated. Ex. Voice Volume of 0 is expected during transitions instead of NO Talking during transitions 4 Procedures/Expectations are kid-friendly. They are written in a manner that students will understand. 5 Voice Volume expectations are indicated for each activity that is addressed. 6 Attention signal(s) are posted. 7 The matrix is posted in a visible place within the classroom

Classroom Matrix Walkthrough Goal May 2015 Nov 2015 April 2016 School-wide expectations Be Safe, Be Respectful, Be Responsible are displayed. 96 100 Procedures/Expectations are identified for at least 4 activities within the classroom. These should include arrival, transitions, cooperative learning groups, and whole group instruction. Specialists should at minimum address 2 areas. 80 Procedures/Expectations should be positively stated. Ex. Voice Volume of 0 is expected during transitions instead of NO Talking during transitions Procedures/Expectations are kid-friendly. They are written in a manner that students will understand 92 Voice Volume expectations are indicated for each activity that is addressed. 68 88 Attention signal(s) are posted. 48 The matrix is posted in a visible place within the classroom 84

Student Interviews Interview Questions #1- What are the expectations for (pick one area noted on matrix)? #2- What is the signal your teacher uses to get your attention? Give example of one in lunchroom (When I say 0 you say shhh) #3- What happens in your classroom when someone does not have expected behavior? Rubrics for Scoring Rubric 2- Student notes at least 2 expectations for targeted area Rubric 2- Student is able to state the attention signal Rubric 2- All three students give a similar response Rubric 1- Students notes 1 expectation for targeted area Rubric 1- When given the first part of the attention signal, the student responds with the student response Rubric 1- Two of the three students give a similar response Rubric 0- Student is unable to provide any expectations for targeted area Rubric 0- Student is unaware of attention signal. Rubric 0- All three students give different responses Student #1 Student #2 Student #3

Student Interview Data Specific Area April/May 2015 January 2016 April/May 2016 Classroom Expectations 81% 92% 100% Awareness of Attention Signal 85% 75% 89% Awareness of Consequences of Unexpected Behavior 49% 36% 68% Overall 72% 67% 86%

Lessons Learned Works well Mistakes made Changes for the future Having a rubric makes for consistent answers Sharing data with teacher Specific feedback Mistakes made 22 new staff… Never assume… Changes for the future Reword question related to consequence systems Have specific items to share with teachers for each area of the rubric to work on