Using data to create equitable systems: Risk ratios and root cause analysis (E8) Emilie O’Connor oconnore@wisconsinpbisnetwork.org Sara Summ summs@wisconsinrticenter.org

Session Description Creating a system that supports all students equitably is a priority for educators. In this session, participants will first explore how risk ratios can help identify the extent of areas of disproportionality. Participants will also learn how to dig deeper and analyze the root causes of disparities so that informed action can be taken. Additional tools and strategies will be provided for further self-reflection and practice. Highlighted in orange the session outcomes

A Model to Inform Culturally Responsive Practices This session employs each aspect of this model Self-awareness: How we enter the work Examine the system’s impact: risk ratio and root cause analysis Believe that all students will learn: how we frame disproportionality Understand the we all have unique identities and world views/know the communities: How we start to learn more about who WE are and who are students are Lead, model… and accept institutional responsibility: How to take on the work to ensure equitable outcomes for all kids Use practices: This process will help reveal what policies & practices we may need to change in order to better serve kids

Agreements Source: National Equity Project Can we commit to… Notice moments of discomfort and stay curious? Listen fully, with our ears, eyes and heart? Speak our truth without blame or judgment? Be open to the experience and each other? Source: National Equity Project

Overall 2014-15 graduation rate (6 year): 92.1% Some Wisconsin data Overall 2014-15 graduation rate (6 year): 92.1% Students with disabilities: 82.0% Economically disadvantaged: 84.3% Limited English Proficiency: 76% Females: 93.8%; Males: 90.5% Migrant: 80.6% American Indian: 80.2% Asian: 95.5% Black: 74.2% Hispanic: 83.2% Pacific Isle: 91.2% White: 95.2% Two or More: 90.7% For whom is this new information? How does this data make you feel? Overwhelmed? Out of one’s control?

Overall 2011-12 graduation rate (6 year): 90.4% Some Wisconsin data Overall 2011-12 graduation rate (6 year): 90.4% Students with disabilities: 78.8% Economically disadvantaged: 81.3% Limited English Proficiency: 77.8% Females: 92.2%; Males: 88.8% Migrant: 77.5% American Indian: 75.3% Asian: 92.1% Black: 71.5% Hispanic: 78.1% White: 94.0% 2042 – time for AA to reach 100% graduation rate at this pace 2037 -> 98% (two generations: 6 year grad rate of 2027 for current grade 1 students) If stay at current rate of 75%, 16,500 AA students currently in system won’t graduate from HS in 6 year cohort (more than people in the city of Oconomowoc or Menomonie) How does this data make you feel? Overwhelmed? Out of one’s control?

Reflection Questions What is the nature of the conversations in team discussions about students and their families? What are our norms? When and where do we disaggregate data? Do we know who is being underserved and to what extent? What are our protocols for discussion? When encountering disproportionate data, do we DESCRIBE AND DEFLECT or REFLECT AND INSPECT? What are our protocols for digging deeper? To what extent do we consider our own cultural influence on student behavior and achievement? On decision making? Who owns the change? Where does this action show up in our PD and improvement plans? Who sits at the table for problem-solving? How do we engage students, families and the community in solution-seeking? Let’s imagine that the data shown was YOUR data…or think about the last team discussion you had around disaggregated data. Consider the questions here…

“Research suggests that more generic consciousness of …inequality can actually be deadening for both educators and students unless analysis pinpoints concrete ways of counteracting” inequities. This research references race specifically, but can be applied across –isms Source: Pollock, M., Deckman, S., Mira, M., & Shalaby, C. ( 2010). "But what can I do?" Three necessary tensions in teaching teachers about race. Journal of Teacher Education, 61, 211-224.

Underperforming Underserving Describe and deflect Inspect and reflect What is the nature of the conversation in your school about disproportionality? Underperforming Underserving When you look at disaggregated data, do conversations tend to be more in the pink or more in the green? Even though the pink is easier, actually creates sense of helplessness described Green puts the power back into the control of the individuals in the school Who owns the change? This is the first tool we want to share with you. Some implementation strategies: Build into your team meeting norms Use to examine the protocols you use in team meetings Initially assign this as a team role (it sounds like we are deflecting, what can we control?) Post statements with norms Describe and deflect Inspect and reflect Anthony Muhammed

When a flower doesn’t bloom you fix the environment in which it grows, not the flower. Alexander Den Heijer Alexander Den Heijer, Amsterdam, leadership Share gardening story about moving rose bush into more sunny environment vs. beneath the maple (didn’t blame the flower shop or spend time questioning the rose’s ability or desire to grow) First, we need to examine the data and see how well our garden is supporting our flowers….

Questions to facilitate exploration of root causes: What is the rate of students suspended by disability? Race? Gender? What is the rate of disciplinary referrals to suspensions by disability? Race? Gender? What is the most common disciplinary referral by academic level (i.e., exam scores, IEP, gifted and talented)? Disability? Race? Gender? Are there differences in suspensions and length of suspensions by academic level (i.e., exam scores, IEP, gifted and talented)? Disability? Race? Gender? Source: NYU Steinhart. Identifying the Root Causes of Disproportionality

1. Problem Identification 2. Problem Analysis 3. Plan Implementation 4. Plan Evaluation 1. Problem Identification Is there a problem? Why is it happening? Main Points: Problem solving method similar to TIPs or Plan Do Study Act. This day will focus mainly on identifying and analyzing the problem. If we attack a problem the same way we’ve always done it, we will get the same result. We need to listen and learn before moving forward with a plan that is based on dominant society. -Plans must include student voice and information from the community in order to have strong contextual fit and resonate with students and families. Plan evaluation is often missing in school teams looking at and planning around data. -School example…. A School that is identified as disproportionate, makes a plan without digging deeper into the data, does not use data to monitor their progress toward their goals, and only knows if they’re making progress based on whether or not they’re re-identified as disproportionate again down the road. Is the plan working? What should be done? (All data use slides adopted from Kelsey Morris; Using Data Presentation; www.pbis.org)

Risk Ratio Relative risk (RR) is the ratio of the probability of an event occurring (for example, developing a disease, being injured) in one group to the probability of the event occurring in a comparison group. A risk ratio is the probability of an event happening when comparing one group with another. Take for example the likelihood of getting a speeding ticket. For this to happen, you have to be driving. The average risk, then, is the likelihood that one person gets a ticket compared to the average. BUT, if you examine if there are certain groups more likely to get a ticket compared to another group, then you have a risk ratio. Men who drive a Volkswagen GTI or Mercedes-Benz CLS-63 AMG are twice as likely to get a ticket than the average driver. If they’re in a Hummer (4.63 times higher), they might as well plan on it. So in this case, male drivers have a risk ratio of 2 (or double the risk) and drivers of a H2 have a risk ratio of 4.6, or almost 5 times the risk of a ticket compared to the average driver. Source of ticket example is http://www.google.com/url?q=http%3A%2F%2Fwww.forbes.com%2F2010%2F10%2F13%2Fcars-that-get-ticketed-most-police-speeding-lifestyle-vehicles-violations.html&sa=D&sntz=1&usg=AFQjCNEDsd2NhWwk3zYmlNLUvaRGiWzMeQ Risk ratio is the first look at equity of outcomes between different enrolled groups. It is used primarily for race and ethnicity comparisons, but can be used for gender, ability status, or even socio/economic status if that information is available. In Wisconsin, that typically means comparing particular educational outcomes for one enrolled subgroups to that same outcome for enrolled White subgroup, as majority enrollment. Then make connection back to Education….

An example of Risk Ratio Risk of Getting a Speeding Ticket Take for example the likelihood of getting a speeding ticket. For this to happen, you have to be driving. The average risk, then, is the likelihood that one person gets a ticket compared to the average. (1st animation) Since we’re comparing to the average driver, the ratio of the average driver getting a speeding ticket is always 1. BUT, if you examine if there are certain groups more likely to get a ticket compared to another group, then you have a risk ratio. Men who drive a Volkswagen GTI or Mercedes-Benz CLS-63 AMG are twice as likely to get a ticket than the average driver., they might as well plan on it. So in this case, male drivers have a risk ratio of 2 (or double the risk) and (2nd and 3rd animation) If they’re in a Hummer (4.63 times higher) drivers of a HummerH2 have a risk ratio of 4.6, or almost 5 times the risk of a ticket compared to the average driver. Source of ticket example is http://www.google.com/url?q=http%3A%2F%2Fwww.forbes.com%2F2010%2F10%2F13%2Fcars-that-get-ticketed-most-police-speeding-lifestyle-vehicles-violations.html&sa=D&sntz=1&usg=AFQjCNEDsd2NhWwk3zYmlNLUvaRGiWzMeQ

Connecting Culturally Responsive Practices To Periodic Data WISCONSIN’S REALITY Suspension (OSS data from Dignity in Schools): Black students are 8 times more likely to be suspended than white students American Indian students are 3 times more likely to be suspended than white students Hispanic students twice as likely to be suspended than white students Academic (2013-14): Hispanic students in grade 3 (Reading ) are 1.45 times more likely to be below benchmark than white students Black students in grade 8 (Mathematics) are 2 times more likely to be below benchmark than white students American Indian and Hispanic students are three times more likely to not graduate high school than white students. We have shared some broad brush strokes about some of our states’ data. We have evidenced from some of the schools that we are working closely with how connecting culturally responsive practices to your periodic data and implementing a responsive framework are reducing suspensions AND office discipline referrals. CONNECTION TO OBJECTIVES: Refer to the slide Work knowledgeably with data, including: analyzing disaggregated results, identifying data gaps, making precision statements, and acknowledging/owning data Closing Summary: (after reading of the slide) Wisconsin’s diversity is a key asset to its public education system, our diversity helps to make businesses more innovative and competitive, and the presence of racial disparities in our educational system pose challenges to opportunities in economic progress.

Mystery Middle School Demographics Ethnicity Number of Students (424 Total) How many received 1 or more tardies Asian 13 3 Black 63 50 Latino 88 45 Native American 8 2 White 252 100

Mystery Middle School Tardy Data for 2015-2016

Calculate Risk Ratios

Calculating Risk 50/63 = 79% 100/252 = 39% 63 African Americans enrolled. Over the past year, 50 African American students received at least 1 tardy. 50/63 = 79% 252 White students are enrolled. Over the past year, 100 White students received at least 1 tardy. 100/252 = 39% Risk Ratio 79/39 = 2 times more likely to receive a tardy

Run a risk ratio Create a problem statement: “The way our system is set up now, xxx students are xxx times more likely to xxx than xxx.” Example: The way Mystery Middle School’s system is set up now, African American students are 2 times more likely to receive a tardy than their White peers. Team planning times: Broad problem identification i.e. dispro in special education; dispro in AP placement Specific details of problem Who, when, how, why is the problem happening? District or School Policies that relate to the identified problem AP requirements such as paper writing, etc.

1. Problem Identification 2. Problem Analysis 3. Plan Implementation 4. Plan Evaluation 1. Problem Identification Is there a problem? Why is it happening? Main Points: Problem solving method similar to TIPs or Plan Do Study Act. This day will focus mainly on identifying and analyzing the problem. If we attack a problem the same way we’ve always done it, we will get the same result. We need to listen and learn before moving forward with a plan that is based on dominant society. -Plans must include student voice and information from the community in order to have strong contextual fit and resonate with students and families. Plan evaluation is often missing in school teams looking at and planning around data. -School example…. A School that is identified as disproportionate, makes a plan without digging deeper into the data, does not use data to monitor their progress toward their goals, and only knows if they’re making progress based on whether or not they’re re-identified as disproportionate again down the road. Is the plan working? What should be done? (All data use slides adopted from Kelsey Morris; Using Data Presentation; www.pbis.org)

Disproportionality has more than one cause and more than one solution Beliefs Practices Policies Disproportionality is the result of the interactions among policies, practices, and beliefs that manifest across educational areas Technical Assistance Center on Disproportionality

Disproportionality is complex Family and community engagement Access and opportunity Learning environment Instruction and assessment Expectations, misconceptions and biases Disproportionality Disproportionality has no one cause but is rather the product of a confluence of contributing factors. These factors provide the conditions and environment in which disproportionate outcomes for students of color occur Use culturally responsive educational systems approach. The approach critically assesses the intersections between policies, practices, and people as they deliver educational services to all students, and considers how these intersections affect disparate outcomes in special education (Klingner et al., 2005). Culturally responsive practices require practitioners to develop a nuanced, reflective, and critical social consciousness and cultural competence about race, power, and privilege in society (Gay, 2000; Ladson-Billings, 1994, 2001). These practices make up a culturally responsive lens through which deficit-oriented beliefs and approaches to educating students are challenged through systems, policies, and people (Harry & Klingner, 2006; Klingner et al., 2005). Using a culturally responsive lens implies that students “can excel in academic endeavors Source: https://steinhardt.nyu.edu/scmsAdmin/media/users/ll81/Identifying_the_Root_Causes_of_Disproportionality.pdf Policies and practices Cultural dissonance Adapted from Identifying the Root Causes of Disproportionality

Variables Over Which School has Control Technical Policies Procedures Programs Schedule Curriculum Instructional methods Staff roles and responsibilities SLOs Staff development Team protocols Imaging Interaction patterns Adaptive Low expectations/fixed mindset Biases, resentments, fears Sense of self-efficacy Knowledge Stereotypes, misconceptions Deficit vs. asset thinking Belief systems Ownership of vision/mission Relationships Connectedness to schooling History with schooling

Problem Why? Technical Adaptive https://www.flickr.com/photos/jimforest/8729576668/sizes/l/ Adaptive Make handout

Our mindset for digging deeper Underperforming Underserving Describe and deflect Inspect and reflect Anthony Muhammed

1. Problem Identification 2. Problem Analysis 3. Plan Implementation 4. Plan Evaluation 1. Problem Identification Is there a problem? Why is it happening? Main Points: Problem solving method similar to TIPs or Plan Do Study Act. This day will focus mainly on identifying and analyzing the problem. If we attack a problem the same way we’ve always done it, we will get the same result. We need to listen and learn before moving forward with a plan that is based on dominant society. -Plans must include student voice and information from the community in order to have strong contextual fit and resonate with students and families. Plan evaluation is often missing in school teams looking at and planning around data. -School example…. A School that is identified as disproportionate, makes a plan without digging deeper into the data, does not use data to monitor their progress toward their goals, and only knows if they’re making progress based on whether or not they’re re-identified as disproportionate again down the road. Is the plan working? What should be done? (All data use slides adopted from Kelsey Morris; Using Data Presentation; www.pbis.org)

Then what? Data: Disproportionality across behavior, math & reading trend data Focus: Demonstrate cultural competence when collaborating in teams about universal / Tier 1 student data and instructional practices (SIR #37) Possible Reasons for Disproportionality Potential Action Steps Stuck on what we can’t control Incorporate CR practices into universal curriculum and instruction Identify resources for CR practices (e.g. Montgomery Schools) Assistance from district staff Lack of awareness (prior to now) Continue conversations in CST/Goal Teams (1x/mo) PD for ALL staff (not just leadership team 7 experiences Staff meetings No access to disaggregated data Current focus on student underperforming v school underserving Build disaggregating data into processes: Data wall MAP/STAR SWIS

Reflection Questions What is the nature of the conversations in team discussion about students and their families? (asset or deficit-focused?) To what extent are we considering our own cultural influence on student behavior and achievement? When encountering at disproportionate data, do teams describe and deflect or reflect and inspect? Who owns the change? Who is sitting at the table? How are we engaging families and the community in solution-seeking? What is the nature of our relationships with families? Add other reflection questions

Additional Resources Wisconsin PBIS Network: Risk Ratio eLearning course http://www.thenetworkwi.com/

Additional Resources US Dept of Ed Safe and Supportive Schools Examples of publications from NYU Steinhardt Dispro Center: Behavioral Support Root Cause Analysis Workbook An Equity Lens for Early Warning Systems: Monthly Calendar for Data Teams Addressing Disproportionality Through the Creation of Culturally Responsive Problem-Solving Teams Identifying the Root Causes of Disproportionality TFI Companion Guide -> Resources -> Publications PBIS.org

Or contact us! Emilie O’Connor Sara Summ oconnore@wisconsinpbisnetwork.org Sara Summ Hand out flower cards summs@wisconsinrticenter.org