2. Using data to create equitable systems: Risk ratios and root cause analysis (E8)
Emilie O’Connor
Sara Summ

3. Session Description
Creating a system that supports all students equitably is a priority for educators. In this session, participants will first explore how risk ratios can help identify the extent of areas of disproportionality. Participants will also learn how to dig deeper and analyze the root causes of disparities so that informed action can be taken. Additional tools and strategies will be provided for further self-reflection and practice.
Highlighted in orange the session outcomes

4. A Model to Inform Culturally Responsive Practices
This session employs each aspect of this model
Self-awareness: How we enter the work
Examine the system’s impact: risk ratio and root cause analysis
Believe that all students will learn: how we frame disproportionality
Understand the we all have unique identities and world views/know the communities: How we start to learn more about who WE are and who are students are
Lead, model… and accept institutional responsibility: How to take on the work to ensure equitable outcomes for all kids
Use practices: This process will help reveal what policies & practices we may need to change in order to better serve kids

5. Agreements Source: National Equity Project Can we commit to…
Notice moments of discomfort and stay curious?
Listen fully, with our ears, eyes and heart?
Speak our truth without blame or judgment?
Be open to the experience and each other?
Source: National Equity Project

6. Overall 2014-15 graduation rate (6 year): 92.1%
Some Wisconsin data
Overall graduation rate (6 year): 92.1%
Students with disabilities: 82.0% Economically disadvantaged: 84.3% Limited English Proficiency: 76% Females: 93.8%; Males: 90.5% Migrant: 80.6%
American Indian: 80.2% Asian: 95.5% Black: 74.2% Hispanic: 83.2% Pacific Isle: 91.2% White: 95.2% Two or More: 90.7%
For whom is this new information?
How does this data make you feel?
Overwhelmed? Out of one’s control?

7. Overall 2011-12 graduation rate (6 year): 90.4%
Some Wisconsin data
Overall graduation rate (6 year): 90.4%
Students with disabilities: 78.8% Economically disadvantaged: 81.3% Limited English Proficiency: 77.8% Females: 92.2%; Males: 88.8% Migrant: 77.5%
American Indian: 75.3% Asian: 92.1% Black: 71.5% Hispanic: 78.1% White: 94.0%
2042 – time for AA to reach 100% graduation rate at this pace
2037 -> 98%
(two generations: 6 year grad rate of 2027 for current grade 1 students)
If stay at current rate of 75%, 16,500 AA students currently in system won’t graduate from HS in 6 year cohort (more than people in the city of Oconomowoc or Menomonie)
How does this data make you feel?
Overwhelmed? Out of one’s control?

8. Reflection Questions
What is the nature of the conversations in team discussions about students and their families? What are our norms?
When and where do we disaggregate data? Do we know who is being underserved and to what extent? What are our protocols for discussion?
When encountering disproportionate data, do we DESCRIBE AND DEFLECT or REFLECT AND INSPECT? What are our protocols for digging deeper?
To what extent do we consider our own cultural influence on student behavior and achievement? On decision making?
Who owns the change? Where does this action show up in our PD and improvement plans?
Who sits at the table for problem-solving? How do we engage students, families and the community in solution-seeking?
Let’s imagine that the data shown was YOUR data…or think about the last team discussion you had around disaggregated data. Consider the questions here…

9. “Research suggests that more generic consciousness of …inequality can actually be deadening for both educators and students unless analysis pinpoints concrete ways of counteracting” inequities.
This research references race specifically, but can be applied across –isms
Source: Pollock, M., Deckman, S., Mira, M., & Shalaby, C. ( 2010). "But what can I do?" Three necessary tensions in teaching teachers about race. Journal of Teacher Education, 61,

10. Underperforming Underserving Describe and deflect Inspect and reflect
What is the nature of the conversation in your school about disproportionality?
Underperforming
Underserving
When you look at disaggregated data, do conversations tend to be more in the pink or more in the green?
Even though the pink is easier, actually creates sense of helplessness described
Green puts the power back into the control of the individuals in the school
Who owns the change?
This is the first tool we want to share with you.
Some implementation strategies:
Build into your team meeting norms
Use to examine the protocols you use in team meetings
Initially assign this as a team role (it sounds like we are deflecting, what can we control?)
Post statements with norms
Describe and deflect
Inspect and reflect
Anthony Muhammed

11. When a flower doesn’t bloom
you fix the environment in which it grows, not the flower.
Alexander Den Heijer
Alexander Den Heijer, Amsterdam, leadership
Share gardening story about moving rose bush into more sunny environment vs. beneath the maple
(didn’t blame the flower shop or spend time questioning the rose’s ability or desire to grow)
First, we need to examine the data and see how well our garden is supporting our flowers….

12. Questions to facilitate exploration of root causes:
What is the rate of students suspended by disability? Race? Gender?
What is the rate of disciplinary referrals to suspensions by disability? Race? Gender?
What is the most common disciplinary referral by academic level (i.e., exam scores, IEP, gifted and talented)? Disability? Race? Gender?
Are there differences in suspensions and length of suspensions by academic level (i.e., exam scores, IEP, gifted and talented)? Disability? Race? Gender?
Source: NYU Steinhart. Identifying the Root Causes of Disproportionality

13. 1. Problem Identification
2. Problem Analysis
3. Plan Implementation
4. Plan Evaluation
1. Problem Identification
Is there a problem?
Why is it happening?
Main Points:
Problem solving method similar to TIPs or Plan Do Study Act. This day will focus mainly on identifying and analyzing the problem.
If we attack a problem the same way we’ve always done it, we will get the same result. We need to listen and learn before moving forward with a plan that is based on dominant society.
-Plans must include student voice and information from the community in order to have strong contextual fit and resonate with students and families.
Plan evaluation is often missing in school teams looking at and planning around data.
-School example…. A School that is identified as disproportionate, makes a plan without digging deeper into the data, does not use data to monitor their progress toward their goals, and only knows if they’re making progress based on whether or not they’re re-identified as disproportionate again down the road.
Is the plan working?
What should be done?
(All data use slides adopted from Kelsey Morris; Using Data Presentation;

14. Risk Ratio
Relative risk (RR) is the ratio of the probability of an event occurring (for example, developing a disease, being injured) in one group to the probability of the event occurring in a comparison group.
A risk ratio is the probability of an event happening when comparing one group with another.
Take for example the likelihood of getting a speeding ticket. For this to happen, you have to be driving. The average risk, then, is the likelihood that one person gets a ticket compared to the average. BUT, if you examine if there are certain groups more likely to get a ticket compared to another group, then you have a risk ratio. Men who drive a Volkswagen GTI or Mercedes-Benz CLS-63 AMG are twice as likely to get a ticket than the average driver. If they’re in a Hummer (4.63 times higher), they might as well plan on it. So in this case, male drivers have a risk ratio of 2 (or double the risk) and drivers of a H2 have a risk ratio of 4.6, or almost 5 times the risk of a ticket compared to the average driver.
Source of ticket example is
Risk ratio is the first look at equity of outcomes between different enrolled groups. It is used primarily for race and ethnicity comparisons, but can be used for gender, ability status, or even socio/economic status if that information is available.
In Wisconsin, that typically means comparing particular educational outcomes for one enrolled subgroups to that same outcome for enrolled White subgroup, as majority enrollment.
Then make connection back to Education….

15. An example of Risk Ratio Risk of Getting a Speeding Ticket
Take for example the likelihood of getting a speeding ticket. For this to happen, you have to be driving. The average risk, then, is the likelihood that one person gets a ticket compared to the average.
(1st animation)
Since we’re comparing to the average driver, the ratio of the average driver getting a speeding ticket is always 1.
BUT, if you examine if there are certain groups more likely to get a ticket compared to another group, then you have a risk ratio. Men who drive a Volkswagen GTI or Mercedes-Benz CLS-63 AMG are twice as likely to get a ticket than the average driver., they might as well plan on it. So in this case, male drivers have a risk ratio of 2 (or double the risk) and
(2nd and 3rd animation)
If they’re in a Hummer (4.63 times higher)
drivers of a HummerH2 have a risk ratio of 4.6, or almost 5 times the risk of a ticket compared to the average driver.
Source of ticket example is

16. Connecting Culturally Responsive Practices To Periodic Data
WISCONSIN’S REALITY
Suspension (OSS data from Dignity in Schools): Black students are 8 times more likely to be suspended than white students American Indian students are 3 times more likely to be suspended than white students Hispanic students twice as likely to be suspended than white students
Academic ( ): Hispanic students in grade 3 (Reading ) are 1.45 times more likely to be below benchmark than white students Black students in grade 8 (Mathematics) are 2 times more likely to be below benchmark than white students American Indian and Hispanic students are three times more likely to not graduate high school than white students.
We have shared some broad brush strokes about some of our states’ data. We have evidenced from some of the schools that we are working closely with how connecting culturally responsive practices to your periodic data and implementing a responsive framework are reducing suspensions AND office discipline referrals.
CONNECTION TO OBJECTIVES: Refer to the slide
Work knowledgeably with data, including: analyzing disaggregated results, identifying data gaps, making precision statements, and acknowledging/owning data
Closing Summary: (after reading of the slide)
Wisconsin’s diversity is a key asset to its public education system, our diversity helps to make businesses more innovative and competitive, and the presence of racial disparities in our educational system pose challenges to opportunities in economic progress.

17. Mystery Middle School Demographics
Ethnicity
Number of Students (424 Total)
How many received 1 or more tardies
Asian 13 3
Black 63 50
Latino 88 45
Native American 8 2
White
252
100

18. Mystery Middle School Tardy Data for 2015-2016

19. Calculate Risk Ratios

20. Calculating Risk 50/63 = 79% 100/252 = 39%
63 African Americans enrolled. Over the past year, 50 African American students received at least 1 tardy.
50/63 = 79%
252 White students are enrolled. Over the past year, 100 White students received at least 1 tardy.
100/252 = 39%
Risk Ratio 79/39 = 2 times more likely to receive a tardy

21. Run a risk ratio
Create a problem statement:
“The way our system is set up now, xxx students are xxx times more likely to xxx than xxx.”
Example: The way Mystery Middle School’s system is set up now, African American students are 2 times more likely to receive a tardy than their White peers.
Team planning times:
Broad problem identification
i.e. dispro in special education; dispro in AP placement
Specific details of problem
Who, when, how, why is the problem happening?
District or School Policies that relate to the identified problem
AP requirements such as paper writing, etc.

22. 1. Problem Identification
2. Problem Analysis
3. Plan Implementation
4. Plan Evaluation
1. Problem Identification
Is there a problem?
Why is it happening?
Main Points:
Problem solving method similar to TIPs or Plan Do Study Act. This day will focus mainly on identifying and analyzing the problem.
If we attack a problem the same way we’ve always done it, we will get the same result. We need to listen and learn before moving forward with a plan that is based on dominant society.
-Plans must include student voice and information from the community in order to have strong contextual fit and resonate with students and families.
Plan evaluation is often missing in school teams looking at and planning around data.
-School example…. A School that is identified as disproportionate, makes a plan without digging deeper into the data, does not use data to monitor their progress toward their goals, and only knows if they’re making progress based on whether or not they’re re-identified as disproportionate again down the road.
Is the plan working?
What should be done?
(All data use slides adopted from Kelsey Morris; Using Data Presentation;

23. Disproportionality has more than one cause and more than one solution
Beliefs
Practices
Policies
Disproportionality is the result of the interactions among policies, practices, and beliefs that manifest across educational areas
Technical Assistance Center on Disproportionality

24. Disproportionality is complex
Family and community engagement
Access and opportunity
Learning environment
Instruction and assessment
Expectations, misconceptions and biases
Disproportionality
Disproportionality has no one cause but is rather the product of a confluence of
contributing factors. These factors provide the conditions and environment in which
disproportionate outcomes for students of color occur
Use culturally responsive educational systems approach. The approach critically assesses the intersections between
policies, practices, and people as they deliver educational services to all students, and considers
how these intersections affect disparate outcomes in special education (Klingner et al., 2005).
Culturally responsive practices require practitioners to develop a nuanced, reflective,
and critical social consciousness and cultural competence about race, power, and privilege in
society (Gay, 2000; Ladson-Billings, 1994, 2001). These practices make up a culturally
responsive lens through which deficit-oriented beliefs and approaches to educating students
are challenged through systems, policies, and people (Harry & Klingner, 2006; Klingner et al.,
2005). Using a culturally responsive lens implies that students “can excel in academic endeavors
Source:
Policies and practices
Cultural dissonance
Adapted from Identifying the Root Causes of Disproportionality

25. Variables Over Which School has Control
Technical
Policies
Procedures
Programs
Schedule
Curriculum
Instructional methods
Staff roles and responsibilities
SLOs
Staff development
Team protocols
Imaging
Interaction patterns
Adaptive
Low expectations/fixed mindset
Biases, resentments, fears
Sense of self-efficacy
Knowledge
Stereotypes, misconceptions
Deficit vs. asset thinking
Belief systems
Ownership of vision/mission
Relationships
Connectedness to schooling
History with schooling

26. Problem Why? Technical Adaptive
Adaptive
Make handout

27. Our mindset for digging deeper
Underperforming
Underserving
Describe and deflect
Inspect and reflect
Anthony Muhammed

28. 1. Problem Identification
2. Problem Analysis
3. Plan Implementation
4. Plan Evaluation
1. Problem Identification
Is there a problem?
Why is it happening?
Main Points:
Problem solving method similar to TIPs or Plan Do Study Act. This day will focus mainly on identifying and analyzing the problem.
If we attack a problem the same way we’ve always done it, we will get the same result. We need to listen and learn before moving forward with a plan that is based on dominant society.
-Plans must include student voice and information from the community in order to have strong contextual fit and resonate with students and families.
Plan evaluation is often missing in school teams looking at and planning around data.
-School example…. A School that is identified as disproportionate, makes a plan without digging deeper into the data, does not use data to monitor their progress toward their goals, and only knows if they’re making progress based on whether or not they’re re-identified as disproportionate again down the road.
Is the plan working?
What should be done?
(All data use slides adopted from Kelsey Morris; Using Data Presentation;

29. Then what?
Data: Disproportionality across behavior, math & reading trend data
Focus: Demonstrate cultural competence when collaborating in teams about universal / Tier 1 student data and instructional practices (SIR #37)
Possible Reasons for Disproportionality
Potential Action Steps
Stuck on what we can’t control
Incorporate CR practices into universal curriculum and instruction
Identify resources for CR practices (e.g. Montgomery Schools)
Assistance from district staff
Lack of awareness (prior to now)
Continue conversations in CST/Goal Teams (1x/mo)
PD for ALL staff (not just leadership team
7 experiences
Staff meetings
No access to disaggregated data
Current focus on student underperforming v school underserving
Build disaggregating data into processes:
Data wall
MAP/STAR
SWIS

30. Reflection Questions
What is the nature of the conversations in team discussion about students and their families? (asset or deficit-focused?)
To what extent are we considering our own cultural influence on student behavior and achievement?
When encountering at disproportionate data, do teams describe and deflect or reflect and inspect? Who owns the change?
Who is sitting at the table? How are we engaging families and the community in solution-seeking? What is the nature of our relationships with families?
Add other reflection questions

31. Additional Resources
Wisconsin PBIS Network: Risk Ratio eLearning course

32. Additional Resources US Dept of Ed Safe and Supportive Schools
Examples of publications from NYU Steinhardt Dispro Center:
Behavioral Support Root Cause Analysis Workbook
An Equity Lens for Early Warning Systems: Monthly Calendar for Data Teams
Addressing Disproportionality Through the Creation of Culturally Responsive Problem-Solving Teams
Identifying the Root Causes of Disproportionality
TFI Companion Guide
-> Resources -> Publications
PBIS.org

33. Or contact us! Emilie O’Connor Sara Summ
Sara Summ
Hand out flower cards