School Growth and Student Growth

School Growth and Student Growth
This presentation looks at two components on the school grading report card – school growth and student growth.

Roadmap Introduction School growth Student growth Comparison/summary
“Proficiency” versus “growth” Growth in accountability School growth Definition School grading report cards Student growth Comparison/summary Statistical modeling For this presentation, we will first take a look at the differences between proficiency and growth and why growth is part of the A-F School Grading accountability system. Then we will zoom in on school growth, which can also be called school improvement, and student growth. What do they mean on the school report cards? We’ll compare the two components and summarize the major points of the presentation. Finally, for those interested in the more technical detail of the computations, how these growth measures are calculated are presented. Let’s get started.

“Proficiency” versus “growth” Growth in accountability School growth Definition School grading report cards Student growth Comparison/summary Statistical modeling In this first section, we will discuss the difference between “proficiency” versus “growth” and why growth is incorporated into our school grading accountability system.

Proficiency vs. Growth Proficiency Growth Definition Status Change
Time points One point in time Two or more time points Data Single snapshot of student achievement Multiple snapshots f student achievement For School Grading, proficiency is a measure of status that captures academic performance at a single point in time. Looking at proficiency answers questions such as how well did Noah do on his math test and is Olivia performing at grade level.

Proficiency vs. Growth Growth models Systematic methods
Definition Status Change Time points One point in time Two or more time points Data Single snapshot of student achievement Multiple snapshots of student achievement Growth models Systematic methods Multiple time points Distinction between two or more time points Growth, in contrast, refers to a change over time. A common analogy is that of a child growing in height. If one measures a child’s current height as 4 feet 2 inches, that is an example of current status. On the other hand, if one indicates that a child has grown 3 inches over the past year, that is an example of growth. Similarly, growth is often used to describe change in academic performance among students over time. Growth models used in school accountability are systematic methods to mathematically, or statistically, describe the change in student academic performance. The essential components of a growth model are multiple time points.

Proficiency vs. Growth Proficiency: 60% to 40% from 2016 to 2017
Growth: Positive growth Diego Scores Proficiency 2015 2016 2017 Lucia Jack Mia Using some general examples, let’s continue to look at how proficiency and growth differ. This graph presents the scores of five students at three points in time: 2015, 2016, and Proficiency is indicated by the dashed line. In 2016, three out of five students – Diego, Lucia, and Jack, or 60%, were proficient. The following year in 2017, only two out of the five students – Diego and Lucia, or 40%, were proficient. So the proficiency rate of this school dropped from 2016 to 2017. Although the school’s proficiency rate declined, let’s consider what’s happening with growth in this example. From 2016 to 2017, four out of five students, everyone except Jack, or 80%, improved their scores, so this school helped its students improve over time. It’s interesting to note that in this example, the school’s proficiency rate dropped, but the school showed positive growth. David

Proficiency vs. Growth Proficiency: 40% to 60% from 2016 to 2017
Growth: Negative growth Scores Proficiency 2015 2016 2017 Ava Ben Carlos This graph shows the opposite situation where the school’s proficiency rate has improved, but its growth has not. In this example, Ava and Ben were above the dashed proficiency line in 2016. The next year Ava, Ben, and Carlos were proficient. Therefore, this school’s overall proficiency increased from 40% (two out of five students) in 2016 to 60% (three out of five students) in 2017. However, in this example, growth is negative because the scores of four out of five students dropped from 2016 to Only Carlos had increasing scores, shown by the green line going up. Dahlia Ethan

Positive Growth vs. Negative Growth
Positive growth: from 2016 to 2017 Negative growth: from 2015 to 2016 Scores Proficiency 2015 2016 2017 Henry Lucia Aliyah Xavier Let’s take a look at another example. This graph shows that all students improved from 2016 to 2017, so growth is positive. However, taking into account two prior years of data, all students’ scores actually dropped from 2015 to Therefore, the growth is negative based on all three time points. Because the School Grading model uses two prior years of data in addition to the current year’s, this school would show negative growth. Jade

Why Growth Models in Accountability?
Static performance measures Growth models Identify school effectiveness Provide equal opportunities to succeed Consider overall student from year to year Are also referred to as value-added models (VAM) Now that we have discussed the difference between proficiency and growth, let’s focus on growth. Why is growth an important component of the school accountability system? In addition to New Mexico, many states incorporate growth models into their accountability systems. Growth models can supplement static proficiency measures, and together they can provide a broader picture of school effectiveness and improvement. While status models measure absolute performance without considering other variables, growth models take into account students’ performance in prior grades and some school characteristics.

Concept of Growth in Accountability
Positive growth Actual Score Residual Expected Score Let’s look at how growth is measured. An easy-to-understand way to measure growth is to calculate the difference between this year’s scores and last year’s scores. If the difference is positive, then this indicates improvement. If the difference is negative, then this indicates a decline in performance. However, because students may take different tests in different school years (such as iStation in the 2nd grade and PARCC in the 3rd grade), and these different tests may be on different scales, and this simple method may not be sufficient. Therefore, School Grading uses statistical modeling that takes into account two years of prior test scores as well as other school characteristics. Here’s an example to help visualize growth computation. This graph shows that a particular student’s score was 80 in school year 2015 and 40 in school year Based on these two prior scores and other characteristics of the student, the growth model predicts that this student would earn an average score of 60 in 2017. The student actually earned a score of 65, which is 5 points higher than the expected score, showing positive growth. Growth indices are calculated based on the differences between the expected scores and the actual scores. In more technical terms, this difference is referred to as the residual, as shown in this graph.

Concept of Growth in Accountability
Negative growth Expected Score Residual In this example, the student’s score was 30 in 2015 and 70 in 2016. Based on these two scores and other characteristics of the student, the growth model predicts that this student would earn a score of 50 in 2017. However, the student’s actual 2017 score was 30, which is 20 points lower than the expected score, showing negative growth. Actual Score

“Proficiency” versus “growth” Growth in accountability School growth Definition School grading report cards Student growth Comparison/summary Statistical modeling At PED, two types of growth are utilized in our A-F School Grading accountability model: school growth and student growth. In the next section, we will focus on school growth and demonstrate its use in school grading.

What is School Growth? Overall student improvement from year to year
Includes every student within a school Can be negative or positive Positive: Better than expected Negative: Worse than expected What do we mean by school growth exactly? This measure answers this question: Is the school as a whole improving academically? School growth compares overall student improvement from year to year and considers the progress of every student within a school whether or not he or she is proficient. For example, even if a student does not score proficient on PARCC from one year to the next, he or she can still demonstrate positive growth. School growth indices can be positive or negative. When the growth index is a positive number above zero, it means that the school is growing more than other schools across the state that were similar in terms of students’ previous test scores and other characteristics such as student mobility and school size. The school performed better than was expected. If school growth is a number very close to zero, it means that the school is growing at the same rate as other similar schools across the state. When growth is a negative number below zero, the school performed worse than expected. On average the school is growing less than other similar schools across the state. This does not mean that the school is not improving. Rather, it means that the school is improving less than other similar schools in the state.

School Grading Report Cards
10 points possible in School Grading 5 points for reading 5 points for math Calculated the same way for elementary schools/middle schools for high schools School Improvement School growth counts up to 10 points toward a school’s overall grade, 5 points for reading and 5 points for math. School growth is computed identically for elementary schools/middle schools and for high schools. Specific numbers for a school’s growth are shown on page 3 of the school grading report card under “School Improvement.” In the example shown, this school’s growth index for reading is -0.33, which is below zero, indicating negative growth. The school received only 1.86 points out of the 5 points possible. However, school improvement in math is higher with a growth index of 1.53, yielding 4.68 points and showing positive growth.

School Improvement 6.54 After growth indices are computed, they are then converted into points. More detail on the steps followed for this conversion is provided in the statistical modeling portion of this presentation. In our example, combining the points earned for reading and math resulted in 6.54 total points for school growth. This total is provided on page 1 of the school grading report card.

“Proficiency” versus “growth” Growth in accountability School growth Definition School grading report cards Student growth Comparison/summary Statistical modeling In this section, we will define student growth and show how it is used in school grading.

What is Student Growth? Individual student achievement
Compared with academic peers Average = one year’s worth of growth Can be positive or negative So what is “student growth”? How is it different from school growth? Student growth represents how much individual student achievement grows compared with other students. Growth for each student is measured in relation to how a particular student scored in the current year compared to his or her academic peers. Academic peers are students who scored about the same in the two prior years in reading and math and had other similar characteristics such as grade level and full academic year (FAY) status. A student who scored the average of his or her academic peer group has made one year’s worth of growth for the purposes of school grading. Like school growth, student growth indices are also calculated based on the differences between expected scores and actual scores. For example, if a school has a positive student growth index, the school’s students are now performing better than expected, whereas a negative index means that these students’ achievement is less than predicted.

This school’s higher-performing students performed better than expected. Let’s look at the school grading report card. Every student’s prior test scores are used to estimate how they would have performed this year. If the growth index is above zero, it means that the student performed better than expected based on his or her academic history. If the growth index is near or equal to zero, this means that the student performed as expected compared to the peer group. If the growth index is below zero, this means that the student performed below expectations, and that he or she is falling behind when compared to his or her academic peers. In this example, this high school’s student growth index for reading scores of the higher-performing students is 3.11, which indicates that its higher-performing students performed better than expected. On the other hand, the school’s student growth index for math scores of the lowest-performing students is -0.56, showing that these students performed worse than expected on math. Detail on how the “higher-performing” group or the “lowest-performing” group of students is determined can be found in the training module entitled “Student Quartile (Q) Status” posted on our website. This school’s lowest-performing students performed worse than expected.

Performed higher than expected On the school grading report card, you will see a table showing each subgroup’s growth in reading and math. As you can see here, the subgroups include females, males, Caucasians, African Americans, Hispanics, Asians, American Indians, economically disadvantaged students, students with disabilities, and English learners. The blue shading indicates that a given subgroup performed higher than expected. In this example, the higher-performing African American students performed higher than expected, shown by the positive growth index of On the other hand, the lowest-performing African American students performed below expectations, as shown by the negative growth index of In order to meet confidentiality requirements, information is not shown for subgroups with fewer than 10 students. In this example, the growth index of the lowest-performing American Indian students is not shown, because there were fewer than 10 students in this subgroup. Performed below expectations Fewer than 10 students

Information on student growth over time is also provided graphically on the school grading report card. In this example, we can see that this particular school is doing very well in terms of the growth in reading of the higher-performing students but is not doing well in terms of growth in math of lowest-performing students. For the past three years, its higher-performing students have been performing better than expected, as indicated by the positive growth index 0.96 in 2015, 0.98 in 2016, and 3.11 in 2017, in reading. However, its lowest-performing students have been performing worse on math tests and are falling behind their peers, as indicated by the negative growth index in 2016 and in 2017.

“Proficiency” versus “growth” Growth in accountability School growth Definition School grading report cards Student growth Comparison/summary Statistical modeling Let’s review the characteristics of school growth and student growth side by side.

Comparison Summary School Growth Student Growth Unit of analysis
Individual student Statistical model Prior test scores, full academic year (FAY), school size, subject area/grade test Prior test scores, full academic year (FAY), alternate test, school size, mobility Total points in School Grading 10 for elementary/ middle schools and for high schools 40 for elementary/ middle schools, 20 for high schools Interpretations Negative growth indicates that a school/student subgroup performed worse than expected. Zero growth indicates that a school/student subgroup performed as expected. Positive growth indicates that a school/student subgroup performed better than expected. School growth is based on the entire school, whereas student growth looks at individual students.

Individual student Statistical model Prior test scores, full academic year (FAY), school size, subject area/grade test Prior test scores, full academic year (FAY), alternate test, school size, mobility Total points in School Grading 10 for elementary/ middle schools and for high schools 40 for elementary/ middle schools, 20 for high schools Interpretations Negative growth indicates that a school/student subgroup performed worse than expected. Zero growth indicates that a school/student subgroup performed as expected. Positive growth indicates that a school/student subgroup performed better than expected. The statistical models for both components are similar but different. The predictors for school growth are each school’s prior test scores, whether students enrolled at the school are considered full academic year status or not, the size of the school, and the grade/subject area test. The predictors for student growth are the individual student’s prior test scores, whether the student is considered to have full academic year status, whether he/she took an alternate assessment, the size of his/her school, and the mobility rate of the school.

Individual student Statistical model Prior test scores, full academic year (FAY), school size, subject area/grade test Prior test scores, full academic year (FAY), alternate test, school size, mobility Total points in School Grading 10 for elementary/ middle schools and for high schools 40 for elementary/ middle schools, 20 for high schools Interpretations Negative growth indicates that a school/student subgroup performed worse than expected. Zero growth indicates that a school/student subgroup performed as expected. Positive growth indicates that a school/student subgroup performed better than expected. In School Grading, the total points for school growth are 10 for all schools, which include 5 points for reading and 5 points for math. The total points for student growth are 40 for elementary and middle schools, which include 20 points for the growth of lowest-performing students and 20 points for the growth of higher-performing students. The total points for high schools are 20, including 10 points for the growth of the lowest-performing students and 10 points for the growth of higher-performing students.

Individual student Statistical model Prior test scores, full academic year (FAY), school size, subject area/grade test Prior test scores, full academic year (FAY), alternate test, school size, mobility Total points in School Grading 10 for elementary/ middle schools and for high schools 40 for elementary/ middle schools, 20 for high schools Interpretations Negative growth indicates that a school/student subgroup performed worse than expected. Zero growth indicates that a school/student subgroup performed as expected. Positive growth indicates that a school/student subgroup performed better than expected. Both school growth and student growth can be interpreted in a similar way. If your growth index is a negative value, this shows that your school or student subgroup performed worse than expected when compared to similar schools or the student subgroup’s academic peers. If your growth index is near or equal to zero, this means that your school or student subgroup performed as expected based on the prior test scores and other characteristics such as mobility and school size. If your growth index is positive, then it shows that your school or student subgroup performed better than expected. By looking at the growth index for different subject areas and for different student subgroups, schools can gain insight on how they are doing to ensure all students improve their academic performance. The rest of this module explains the statistical modeling used to calculate growth. Contact information for any questions about school or student growth can be found on the last slide.

“Proficiency” versus “growth” Growth in accountability School growth Definition School grading report cards Student growth Comparison/summary Statistical modeling Finally, let’s take a closer look at the computations for school and student growth.

Statistical Modeling: School Growth
Predicts the average test scores for a school Average values of students’ prior scores School characteristics School size (number of students) Mobility A multiple linear regression model The unit of analysis: the school, not the students School growth predicts the average test scores for a school from the average values of students’ prior scores along with school characteristics including number of students and mobility. The calculation of school growth uses a multiple linear regression model with the unit of analysis being the school, not the students. A multiple linear regression is a linear approach for modeling the relationship between students’ previous test scores, their characteristics such as mobility, and their current test scores. If you have any questions regarding linear regression models, please do not hesitate to contact us.

Statistical Modeling: School Growth
CurrentScore = B0 + B1*Prior1Score + B2*Prior2Scorej + B3*FAY + B4*N + B5*TestName + e CurrentScore: Average score of a school for the current school year B0: Intercept, statewide average B1, B2, B3, B4, & B5: slopes, different weights Prior1Score & Prior2Score: Two prior average scores of the school FAY: Whether or not the students enrolled in the school are considered full academic year status (FAY) N: Size of the school TestName: Test code (MAT03, ALG01, ELA10, etc.) e: Residual School characteristics Here is the regression model that is used to calculate school growth. In the model, CurrentScore is the average score for a school on a given test for the current school year. B0 is the intercept, which is also the statewide average score for the test. B1, B2, B3, B4, and B5 are slopes, which are used to weight different variables in the model. Prior1Score and Prior2Score are two prior average scores for the same test that came from any four prior years, the most recent valid scores available. School characteristics included in the model are the percentage of students who are considered full academic year (FAY) and size of the school (N), which is determined by the number of students tested. TestName refers to test code, which indicates the subject area of a particular assessment. The broader concept of reading or math each contains domains such as third-grade mathematics (MAT03), Algebra I (ALG01), or tenth-grade English Language Arts (ELA10). These areas are applied during modeling to ensure that students are being compared to their academic peers. The important result is the residual. The residual for each test is weighted by the counts of tests, which are then aggregated by TestName and averaged for school growth.

Calculating School Growth Points
Standardization: z scores Transform z scores into a probability that can range from 0 to 1: SchoolGrowthScore = CDF(z) Multiply SGScore by the maximum number of points for the indicator (5 points for each subject) Reading: 5*SchoolGrowthScore Math: 5*SchoolGrowthScore Added together To calculate school growth points, we follow three steps to convert residuals output by the statistical model into points for school grading. We first standardize the growth residuals, which will give a z score for each school. If a school’s z score is 0, then that school’s score is the average score of all schools in the state. The second step is to transform standardized residual values, which are z scores, into probabilities ranging from 0 to 1 using the cumulative normal distribution function. If a school’s z score is 0, then its z score will be converted into a probability of 0.5. The last step is to multiply each school’s school growth score by the maximum numbers of points for the indicator, which is 5 points for each subject. After the number of points is computed separately for reading and math, these points are added together to determine the school growth points each school will receive for school grading.

Statistical Modeling: Student Growth
Calculated separately Lowest-performing quarter (25%) of students Higher-performing three quarters (75%) of students Weighted equally Mixed effects regression model Conducted for each grade/subject area test Aggregated using a weighted average High schools: = 20 Elementary schools and middle schools: = 40 Divided equally between reading and math Student growth is calculated separately for the lowest performing 25% of all students enrolled in a given school and the higher performing 75% of all students enrolled in that school. The points for these two groups are weighted equally, but because there are fewer students in the lowest-performing group than there are in the higher-performing group, the students in the lowest-performing group have more influence, per student, on this portion of the school’s overall points. Student growth is estimated with a mixed effects regression model that is conducted for each test for the two groups separately. The value-added scores for each assessment are aggregated using a weighted average based on the number of students taking the test. The high school model yields a total of 20 points, with 10 points for each student group. The elementary school/middle school model yields a total of 40 points, with 20 points for each student group. Each set of points is divided equally between reading and math.

Statistical Modeling: Student Growth
CurrentScore = B0 + B1*Prior1Score + B2*Prior2Score + B3*FAY + B4*ALT + B5*N + B6*MOB + u + e CurrentScore: Current score of a student B0: Intercept, statewide average B1, B2, B3, B4, B5, & B6: Slopes, different weights Prior1Score & Prior2Score: The student’s two prior scores FAY: Whether or not the student is full academic year status (FAY) ALT: Whether or not the student took an alternate assessment N: Size of the student’s school MOB: Percent of students in the school who are not FAY (mobile) u: School-level effect e: Student-level effect The method for calculating student growth points and school growth points are the same. Here is the mixed effects regression model that is used to calculate student growth. In the model, CurrentScore is the actual score of a student on a given test for the current school year. As with the previous model for school growth, B0 is the intercept and also the average score of all students in the state. B1, B2, B3, B4, B5, and B6 are slopes, which are used to weight different variables in the model. Prior1Score and Prior2Score are the student’s two prior subject scores that came from any four prior years, the most recent valid scores available. Prior scores were derived within the same content area (reading or math). FAY indicates whether the student is considered to have full academic year status or not. ALT indicates whether or not the student took an alternate assessment. This test is the New Mexico Alternate Performance Assessment (NMAPA) given to special education students with the most significant cognitive disabilities. N indicates the size of the school and is represented by the number of students tested. MOB indicates the percent of students in the school who are not considered to have full academic status. Finally, the model also includes a school-level effect and a student-level effect, which represent each student’s and each school’s unique contribution. Although the statistical models are different for school growth and student growth, the method for converting residuals to points for school grading are the same.

For additional information, please contact
Questions? For additional information, please contact Yun Yao, Ph.D., Statistician Ryan Tolman, Ph.D., Statistician For questions about school growth or student growth in the A-F School Grading system, please contact Yun Yao or Ryan Tolman at the addresses shown. Thank you for your attention.

School Growth and Student Growth

Similar presentations

Presentation on theme: "School Growth and Student Growth"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

School Growth and Student Growth

Similar presentations

Presentation on theme: "School Growth and Student Growth"— Presentation transcript:

Similar presentations

About project

Feedback