Understanding How the Ranking is Calculated 2011 Top to Bottom Ranking Understanding How the Ranking is Calculated

Top to Bottom (TTB) Ranking The statewide top-to-bottom ranking takes into account both student achievement on state tests and graduation rates. Student achievement on state tests is included in the statewide top to bottom ranking in the following three ways: Achievement at the elementary, middle, and high school levels Improvement in achievement over time The largest achievement gap between two subgroups calculated based on the top scoring 30% of students versus the bottom scoring 30% of students Rationale for the ranking and some background information that will put it into perspective. So, one critical piece to understand is that in the national and statewide discussion about school performance, the focus has shifted from identifying schools who fail to meet a set of criteria to identifying the schools that are persistently low performing, in order to target resources and interventions more specifically to those schools most in need of assistance.  To help identify these schools, the Michigan Department of Education, or MDE, developed a statewide Top to Bottom ranking.  All k-12 public schools are ranked based on student proficiency, student improvement, and achievement gaps between students within a school.  A school with a high ranking is one that has a high level of proficiency, is improving over time, and is ensuring that all students are learning and achieving at a high level.  This ranking helps to identify those low performing schools that would benefit from additional assistance. It also provides important information to all schools regarding their performance and how it compares to other schools similar to themselves.   So let’s get into the components and the calculations behind the ranking.

TTB Ranking In addition to the achievement components, student graduation is included in the statewide top-to-bottom ranking for schools with a graduation rate in the two following ways: Graduation Rate Improvement in graduation rate over time

Who receives a ranking? Schools with 30 or more students tested over the last two years in at least two state-tested content areas: Mathematics Reading Science Social Studies Writing

How Is the Top to Bottom Ranking Calculated For grade 3-8 reading and mathematics Two-Year Average Standardized Student Scale (Z) Score School Achievement Z-Score 1/2 Two-Year Average Performance Level Change Index School Performance Level Change Z-Score School Content Area Index Content Index Z-score 1/4 The brackets show how the index is created for reading and mathematics in grades 3-8.   As shown at the top left, a standardized scale score is created for each student taking a test. A standardized scale score comes from calculating the student z-score of each student against all students statewide who take the same test in the same grade level in the same year. This assures that each student is compared only to students taking the same type of test in the same year and grade level. The average standardized scale score for each school is then calculated across the two most recent years. Following the arrow to the right, a school achievement z-score is calculated by comparing the school’s two-year average standardized scale score to all other schools in the state on that content area. That quantity is then multiplied by ½ to contribute to the overall school index in the content area. As shown in the middle box on the left side of the brackets, a two-year average performance level change index is calculated using scores displayed in the chart on the next slide. For each the school , the performance level change scores are summed across students and an average is taken to create the two-year average performance level change index. The index for each school is then compared to the rest of the schools in the state to create a school-level performance level change z-score. That z-score is then multiplied by ¼ to contribute to the overall school index in the content area. Finally, as shown on the bottom left of Figure 1, a two-year average bottom 30% minus top 30% z-score gap is created by obtaining the average z-scores of the bottom 30% of z-scores in the school and subtracting from that the average of the top 30% of z-scores in the school. This gives a negative number which when compared to all schools in the state assures that schools with the highest achievement gap receive the lowest z-scores as intended. The school z-score for achievement gap is then multiplied by 1/4 to contribute to the overall school index in the content area.   Combining these elements together creates a school content area index. This is then translated into a final z-score, in order to compare that school’s content area index to other elementary/middle schools or other high schools. Two-Year Average Bottom 30% - Top 30% Z-Score Gap School Achievement Gap Z-Score 1/4

Weighted Performance Level Change A weighted composite of individual student performance level change is used to calculate improvement in grades 3-8 reading and mathematics Rewards large improvements more heavily, rewards maintenance of proficiency if a student was already proficient Previous Proficiency Significant Decline Decline Maintain Improvement Significant Improvement Not Previously Proficient -2 -1 1 2 Previously Proficient The weighted performance level change metric is designed to heavily reward significant improvements, reward improvements, reward maintenance of performance level for students who were already proficient, and disincentivize all declines and significant declines. A ceiling clause is also implemented here such that any student who declines in performance level but remains in the top performance level can be considered to have maintained his or her performance level.

How Is the Top to Bottom Ranking Calculated For science, social studies, writing, and grade 11 reading and mathematics Two-Year Average Standardized Student Scale (Z) Score School Achievement Z-Score 1/2 Four-Year Achievement Trend Slope School Performance Achievement Trend Z-Score School Content Area Index Content Index Z-score 1/4 This slide shows how the index is created for science, social studies, writing in all grades; and for reading and mathematics in grade 11. The only difference between this slide and the previous slide is that rather than performance level change, a four-year achievement trend slope is calculated by regressing two-year average z-scores on school year. This improvement slope is then compared to the improvement slopes for all other schools to derive a school performance achievement trend z-score, which is then multiplied by ¼ to contribute to the overall school index in the content area.   Combining these elements together creates a school content area index. This is then translated into a final z-score, in order to compare that school’s content area index to other elementary/middle schools or other high schools. Two-Year Average Bottom 30% - Top 30% Z-Score Gap School Achievement Gap Z-Score 1/4

How Is the Top to Bottom Ranking Calculated For graduation rate Two-Year Average Graduation Rate School Graduation Rate Z-Score 2/3 Four-Year Graduation Rate Trend Slope School Graduation Rate Trend Z-Score School Graduation Rate Index Grad Index Z-score 1/3 This slide shows how the index for graduation rate is created for schools that have a graduation rate.   Starting at the top left of the schematic, the two-year average graduation rate is calculated, and compared to all other schools’ graduation rates to create a school graduation rate z-score. That z-score is multiplied by 2/3 to contribute to the school graduation rate index. Moving to the bottom left of the schematic, a four year graduation rate trend slope (or annual improvement rate) is calculated by regressing graduation rate on year. That slope is then transformed into a z-score by comparing the school’s slope to the slopes of all other schools. That graduation rate trend z-score is then multiplied by 1/3 to contribute to the school graduation rate index.

How Is the Top to Bottom Ranking Calculated Calculating a four-year slope (e.g., graduation rate) Plot the school’s graduation rate for the last four years Plot a linear regression line through the points Calculate the slope of the line (gives the school’s annual improvement rate) The following three slides demonstrate how a four year slope is calculated. First, you plot the school’s values from each year. In this slide, graduation rate is highlighted.

How Is the Top to Bottom Ranking Calculated Calculating a four-year slope (e.g., graduation rate) Plot the school’s graduation rate for the last four years Plot a linear regression line through the points Calculate the slope of the line (gives the school’s annual improvement rate) Next, you plot a linear regression line through the plots.

How Is the Top to Bottom Ranking Calculated Calculating a four-year slope (e.g., graduation rate) Plot the school’s graduation rate for the last four years Plot a linear regression line through the points Calculate the slope of the line (gives the school’s annual improvement rate) Slope = 2.3% The final step is to calculate the slope of that line. This gives you the school’s annual improvement rate. In the example shown, the school has a slope of 2.3%, which means they have improved an average of 2.3% each year of the four previous years.

How Is the Top to Bottom Ranking Calculated Calculating an overall ranking for a school with a graduation rate School Mathematics Std Index 18% School Reading Std Index 18% School Science Std Index Overall Standardized School Index Overall School Percentile Rank 18% All of the standardized school content area indices and the standardized graduation rate index, where applicable, are then combined to create an overall school index as shown in the slide above.   To create an overall standardized school index, the school standardized graduation rate index is multiplied by 10% to contribute to the overall index. The remaining 90% is equally divided among the content areas for which the school has an index. In the case presented above, for example, the remaining 90% is divided five ways to account for the five content areas in which the school has an index. School Social Studies Std Index 18% School Writing Std Index 18% School Graduation Rate Std Index 10%

How Is the Top to Bottom Ranking Calculated Calculating an overall ranking for a school without a graduation rate School Mathematics Std Index 20% School Reading Std Index 20% School Science Std Index Overall School Standardized Index Overall School Percentile Rank 20% For schools without a graduation rate, 100% of the overall school index is divided equally among the content areas for which the school has an index. For example, in the case of a school that has assessment results in five content areas, each would be weighted 20% School Social Studies Std Index 20% School Writing Std Index 20%

How Is the Top to Bottom Ranking Calculated Calculating an overall ranking for a school without a graduation rate and without a writing score School Mathematics Index 25% School Reading Index 25% School Science Index Overall School Standardized Index Overall School Percentile Rank 25% In the case of a school with results from four content areas, each would be weighted 25%. School Social Studies Index 25%

Resources to Understand my Ranking

Resources Available Complete list of all schools and their ranking Individual school look-up to see your school’s results Overview presentation with voice over FAQ Business rules by which the rankings were calculated Complete data file and validation file You can access these resources at: http://www.michigan.gov/mde/0,4615,7-140-37818_56562---,00.html You can also request individual assistance by calling the Evaluation, Research and Accountability unit at 517-373-1342, or emailing mde-accountability@michigan.gov

What is a Z-Score? Quick Reference for Z-Scores

Z-Score “Cheat Sheet” Student z-score = Z-scores are a standardized measure that helps you compare individual student (or school) data to state average data. Student z-score = (Student Scale Score) – (Statewide average of scale scores) Standard Deviation of Scale Score School z-score= (School Value) – (Statewide average of that value) Standard deviation of that value

Z-Score “Cheat Sheet” Z-scores are centered around zero Positive numbers mean the student or school is above the state average Negative numbers mean the student or school is below the state average State Average …Worse than state average Better than state average…. -3 -2 -1 1 2 3

Z-Score Examples Your school has a z-score of 1.5. You are better than the state average. Z-score of 1.5 State Average …Worse than state average Better than state average…. -3 -2 -1 1 2 3

Z-Score Examples Your school has a z-score of .2. You are better than the state average, but not by a lot. Z-score of 0.2 Z-score of 1.5 State Average …Worse than state average Better than state average…. -3 -2 -1 1 2 3

Z-Score Examples Your school has a z-score of -2.0. You are very far below state average. Z-score of -2.0 Z-score of 0.2 Z-score of 1.5 State Average …Worse than state average Better than state average…. -3 -2 -1 1 2 3

How is this different From the PLA list? TTB vs PLA

TTB versus PLA This list represents a ranking of all schools in the state of Michigan, using our preferred methodology developed in collaboration with many stakeholders. MDE also published, according to state statute, a list of Persistently Lowest Achieving schools. This is the PLA list. The PLA list of schools was generated by a set of federally-approved and required rules that differ from our Top to Bottom ranking.

Why are the lists different? Top to Bottom PLA Subjects included Math Reading Writing Science Social Studies Graduation rate? Yes No Components Achievement (1/2) Improvement (1/4) Achievement gap (1/4) Proficiency (2/3) Improvement (1/3) Proficiency? Uses standardized measure of student performance (z-score) Uses proficiency levels High achieving schools? Calculation adjustments to avoid “ceiling effects” No adjustment Tiers? No tiers; all schools included Tiers; Title I, AYP and school level considered