Upper Darby School District Growth Data Pennsylvania Value Added Assessment System (PVAAS) Developed and maintained by SAS Institute, Inc. District Data Individual School Data In this high-stakes assessment and accountability climate, we need accurate information to guide us in making appropriate decisions for students…decisions about instruction, curriculum, and assessment. PVAAS is one tool to assist schools in this process. PVAAS is an analysis of existing assessment data (PSSA) – no other testing is necessary. PVAAS utilizes longitudinal PSSA test scores to produce reliable estimates of growth. PVAAS is a tool to provide a measure of a school’s effectiveness from year to year. It provides progress data to add to achievement data as schools are analyzing their educational programs for students. PVAAS offers powerful diagnostic information about school practice by leveraging student achievement data with complex statistical tools. It provides a tool to identify strengths and weaknesses in educational programs.

Growth Data vs. Achievement Data What is PVAAS? Analysis of existing assessment data Longitudinal PSSA scores in available content areas Math, Reading, Science, Writing Provides reliable estimates A measure of a school’s effectiveness Critical Relationship: Growth Data vs. Achievement Data In this high-stakes assessment and accountability climate, we need accurate information to guide us in making appropriate decisions for students…decisions about instruction, curriculum, and assessment. PVAAS is one tool to assist schools in this process. PVAAS is an analysis of existing assessment data (PSSA) – no other testing is necessary. PVAAS utilizes longitudinal PSSA test scores to produce reliable estimates of growth. PVAAS is a tool to provide a measure of a school’s effectiveness from year to year. It provides progress data to add to achievement data as schools are analyzing their educational programs for students. PVAAS offers powerful diagnostic information about school practice by leveraging student achievement data with complex statistical tools. It provides a tool to identify strengths and weaknesses in educational programs.

What PVAAS is not… Is not another test Critical concept Is not a comparison of single test scores PVAAS is not another test. PVAAS uses existing assessment data, specifically the PSSA in its statistical models to yield measures of growth. The PSSA is the ONLY common assessment used by all public districts and schools in Pennsylvania. If there are other assessments in the future that all public districts and schools use, they could potentially be added to the PVAAS model. PVAAS measures growth by comparison of estimates based on multiple test scores. Comparison of 2 single test scores such as scaled scores are snapshots . This is NOT an effective way to measure growth.

Achievement vs. Progress The final result of an academic experience Highly correlated with demographic factors, such as socioeconomic factors outside of school Measures students’ performance at a single point Compares student performance to a standard Critical to students post- secondary opportunities Is the concept underlying value-added analysis and reporting Not correlated with demographic factors Dependent upon what happens as a result of schooling Measures students’ progress across time/years Compares student performance to his/her own prior performance Critical to ensuring students’ future academic success Let’s look at how we have traditionally viewed school achievement. We give a summative assessment, usually at the end of the school year, and see whether or not students are measuring up to end of year expectations, without taking into account their starting point. By looking at end of year performance, we can conclude that some students, schools or districts are “low,” “average,” or “high” in terms of student achievement. However, this does not provide the entire picture as we are trying to make data-informed decisions in our schools and districts. This does not tell us about student growth, or progress, that may have occurred during that school year – because we are not taking the starting point into consideration. PVAAS DOES take into account the starting point. PVAAS is a different way of looking at the data that adds the critical dimension of growth, or progress. It is not another test but a comprehensive analysis of achievement trends and progress rates. With PVAAS we are able to look at BOTH progress AND the level of achievement at a given point in time. PVAAS is able to provide a measure that illustrates what effect “schooling” has had on students, regardless of variables such as ethnicity, socio-economic status, ESL and disability. PVAAS indicates the growth, or progress, these groups of students have made between state assessments (PSSA) on an annual basis. PVAAS is the only PDE data tool that provides these measures of growth.

TWO Types of PVAAS Information Looking Forward/Planning… PVAAS Projection Reports For Individual Students and Groups of Students Looking Back/Evaluation… Value-added Growth Reports For Groups of Students There are two main types of data from PVAAS: The Value-added, or growth, information “looks back”…it helps schools to evaluate the effectiveness of the school. How much growth did students make in the past school year? The Projection information “looks forward”…it helps schools plan for the future for students. Are students on a path to proficiency or higher? Both serve different purposes, and both are equally important for continuous school improvement. We are going to start by discussing the value-added growth reports- and how measures of growth are determined in Pennsylvania. Today

Growth Standard Methodology Math & Reading – Grades 4 through 8 Definition: A group makes one year’s growth when they maintain their relative achievement level from one year to the next in relation to the statewide distribution (2006 baseline).  Let’s start with the Growth Standard Methodology for grades 4-8, reading and math. It is about maintaining the position of a group of students in the statewide distribution of test scores from one grade level to the next. So what does that mean??? Is there a new target each year???

NCE (Normal Curve Equivalent) Units A Challenge Comparing Scores Across Years Math & Reading – Grades 4 through 8 1000 1100 1200 1300 1400 1500 Scaled Score Units 0 10 20 30 40 50 60 70 80 90 100 NCE (Normal Curve Equivalent) Units Next comes the challenge of how to compare scores over time. The averages and distribution of scores of PSSA exams change annually. This is normal and expected. So PSSA scaled scores from different years for different grades cannot be compared grade to grade. It is necessary to convert PSSA scaled scores to a common scale for grades 4 through 8. PVAAS rescales each of the PSSA distributions from each year using a Normal Curve Equivalent Scale (NCE) – a common scale for comparisons over time. NCEs are a common language for the scores to look at growth over time. Once a set of test scores has been rescaled, they now can be compared across different grades and years. PVAAS reporting for grades 4 through 8, reading and math uses NCE units to allow for these kinds of comparisons for the calculation of longitudinal growth. These conversions to NCEs are based on the base year of 2006--- so the answer to the one question is I recently asked….does the target change each year? - the answer is NO! The distributions from 2006 are used to determine position of a group to themselves- just like the height charts in the doctors office. They were established in a particular year- then each year you can compare a child to himself/herself to see if they have changed position- the same is true for academic growth as measures by PVAAS for grades 4-8, reading and math. August 2011

PVAAS Growth Measure ≈ Change in position from 5th to 6th grade What Value-Added Growth Measure Means! Growth Standard Methodology Math & Reading – Grades 4 through 8 Achievement 5th grade 47.6 Achievement 6th grade 52.7 This slide displays what the Value-added Growth Measure means. If you are into statistics you may want to know that the NCE Scale sets the State Mean at 50 with a standard deviation of 21.06 so that nearly all data will fall within the range of 0 to 100. It is important to remember that this is simply about rescaling so that scores from different years (different scales) can be compared! [CLICK] Now, assume that a group of students has an estimated achievement at the end of 5th grade that equals 47.6. (Remember, the estimated achievement value of 47.6 is not based on one score; it is an estimate based on all longitudinal data and has been converted to NCEs) Suppose that the group of students estimated achievement at the end of 6th grade was 52.7. We will then complete the displayed calculation to estimate a growth of 5.1 NCE units for the 6th grade school year. The group improved their position by 5.1 NCE units. This means that the school has fostered an improvement in position for these students by 5.1 NCE units from the previous year. While this is a simple explanation of the concept behind PVAAS, it is a much more robust statistical process than simply comparing 2 different scores. 0 10 20 30 40 50 60 70 80 90 100 NCE Scale State Mean = 50 PVAAS Growth Measure ≈ Change in position from 5th to 6th grade ≈ 52.7 – 47.6 ≈ 5.1 NCE units August 2011

Predictive Methodology Math & Reading – Grades 11 Writing – Grades 5, 8, 11 Science – Grades 4, 8, 11 Definition: A group meets the growth measure when their actual achievement meets their predicted achievement - based on the average schooling experience. The Predictive Methodology is about comparing the observed mean to the predicted mean and determining if the group of students performed as expected. The process for determining growth for grade 11 reading and math, as well as science and writing is as follows: A predicted score is calculated for each student; Then a Mean or average of the predicted scores is calculated for the group of students; Then the student actually take the PSSA, and a mean of observed scores for the group of students is calculated. The growth measures is determined by comparing the observed mean to the predicted mean. Did the group meet, exceed or not meet the predicted mean- what was expected. Every measure has error- therefore a standard error is applied as well which you will see on the reports. This is how growth is calculated for grade 11 reading and math, as well as science and writing.

What the colors on the Value Added graph signify… DB Significant evidence that the district/school exceeded the standard for PA Academic Growth LB Moderate evidence that the district/school exceeded the standard for PA Academic Growth G Evidence that the district/school met the standard for PA Academic Growth Y Moderate evidence that the district/school did not meet the standard for PA Academic Growth R Significant evidence that the district/school did not meet the standard for PA Academic Growth The Ratings on the PVAAS Value-added District/School Reports are color-coded to assist you with quick recognition of the rating. While some people want to know where all of the numbers come from and how they are calculated, most people want to know what the report means and how to use it for local decision-making. To provide more information to schools about their effectiveness in meeting the needs of groups of students, this new color legend is being introduced in Fall 2011 that will assist schools in differentiating their growth in both directions…more and less than the standard for PA Academic growth. Dark Blue (DB) - Significant evidence that the district/school exceeded the standard for PA Academic Growth Light Blue (LB) – Moderate evidence that the district/school exceeded the standard for PA Academic Growth Green (G) – Evidence that the district/school met the standard for PA Academic Growth Yellow (Y) - Moderate evidence that the district/school did not meet the standard for PA Academic Growth Red (R) – Significant evidence that the district/school did not meet the standard for PA Academic Growth

District Value Added - Math This report (Value-Added Summary Report) displays the growth status of all schools and grade levels 4-8 in reading and math, in the selected subject. The values are the Growth Measures and the cells are color coded just as they are on the Value-added report. This report is helpful to get a quick view across schools in a district. This report provides the district with an overview of the Growth Measures for all schools and grade levels to identify strengths and weaknesses in growth across buildings and grade levels.

District Value Added - Reading This report (Value-Added Summary Report) displays the growth status of all schools and grade levels 4-8 in reading and math, in the selected subject. The values are the Growth Measures and the cells are color coded just as they are on the Value-added report. This report is helpful to get a quick view across schools in a district. This report provides the district with an overview of the Growth Measures for all schools and grade levels to identify strengths and weaknesses in growth across buildings and grade levels.

District 4th grade Math This report provides evidence of how effective is the District at making academic progress with groups of students at different predicted achievement/performance levels. This is the grade level Performance Diagnostic Report. This report can be generated at a district or school level. The purpose of this report is to provide insight into the growth of students who are predicted to perform at the Below Basic, Basic, Proficient or Advanced levels on the current year’s exam in this subject. This prediction is based on the students’ previous performances. The report consists of two sections – a graph and a table. On the graph there are blue bars representing the growth of the predicted PSSA performance group over the last tested year. The gold bars represent the growth of the three prior years’ groups of students in this predicted PSSA performance level.

District 4th grade Reading This report provides evidence of how effective is the District at making academic progress with groups of students at different predicted achievement/performance levels. This is the grade level Performance Diagnostic Report. This report can be generated at a district or school level. The purpose of this report is to provide insight into the growth of students who are predicted to perform at the Below Basic, Basic, Proficient or Advanced levels on the current year’s exam in this subject. This prediction is based on the students’ previous performances. The report consists of two sections – a graph and a table. On the graph there are blue bars representing the growth of the predicted PSSA performance group over the last tested year. The gold bars represent the growth of the three prior years’ groups of students in this predicted PSSA performance level.

District 5th grade Math This report provides evidence of how effective is the District at making academic progress with groups of students at different predicted achievement/performance levels. This is the grade level Performance Diagnostic Report. This report can be generated at a district or school level. The purpose of this report is to provide insight into the growth of students who are predicted to perform at the Below Basic, Basic, Proficient or Advanced levels on the current year’s exam in this subject. This prediction is based on the students’ previous performances. The report consists of two sections – a graph and a table. On the graph there are blue bars representing the growth of the predicted PSSA performance group over the last tested year. The gold bars represent the growth of the three prior years’ groups of students in this predicted PSSA performance level.

District 5th grade Reading This report provides evidence of how effective is the District at making academic progress with groups of students at different predicted achievement/performance levels. This is the grade level Performance Diagnostic Report. This report can be generated at a district or school level. The purpose of this report is to provide insight into the growth of students who are predicted to perform at the Below Basic, Basic, Proficient or Advanced levels on the current year’s exam in this subject. This prediction is based on the students’ previous performances. The report consists of two sections – a graph and a table. On the graph there are blue bars representing the growth of the predicted PSSA performance group over the last tested year. The gold bars represent the growth of the three prior years’ groups of students in this predicted PSSA performance level.

District 6th grade Math This report provides evidence of how effective is the District at making academic progress with groups of students at different predicted achievement/performance levels. This is the grade level Performance Diagnostic Report. This report can be generated at a district or school level. The purpose of this report is to provide insight into the growth of students who are predicted to perform at the Below Basic, Basic, Proficient or Advanced levels on the current year’s exam in this subject. This prediction is based on the students’ previous performances. The report consists of two sections – a graph and a table. On the graph there are blue bars representing the growth of the predicted PSSA performance group over the last tested year. The gold bars represent the growth of the three prior years’ groups of students in this predicted PSSA performance level.

District 6th grade Reading This report provides evidence of how effective is the District at making academic progress with groups of students at different predicted achievement/performance levels. This is the grade level Performance Diagnostic Report. This report can be generated at a district or school level. The purpose of this report is to provide insight into the growth of students who are predicted to perform at the Below Basic, Basic, Proficient or Advanced levels on the current year’s exam in this subject. This prediction is based on the students’ previous performances. The report consists of two sections – a graph and a table. On the graph there are blue bars representing the growth of the predicted PSSA performance group over the last tested year. The gold bars represent the growth of the three prior years’ groups of students in this predicted PSSA performance level.

District 7th grade Math This report provides evidence of how effective is the District at making academic progress with groups of students at different predicted achievement/performance levels. This is the grade level Performance Diagnostic Report. This report can be generated at a district or school level. The purpose of this report is to provide insight into the growth of students who are predicted to perform at the Below Basic, Basic, Proficient or Advanced levels on the current year’s exam in this subject. This prediction is based on the students’ previous performances. The report consists of two sections – a graph and a table. On the graph there are blue bars representing the growth of the predicted PSSA performance group over the last tested year. The gold bars represent the growth of the three prior years’ groups of students in this predicted PSSA performance level.

District 7th grade Reading This report provides evidence of how effective is the District at making academic progress with groups of students at different predicted achievement/performance levels. This is the grade level Performance Diagnostic Report. This report can be generated at a district or school level. The purpose of this report is to provide insight into the growth of students who are predicted to perform at the Below Basic, Basic, Proficient or Advanced levels on the current year’s exam in this subject. This prediction is based on the students’ previous performances. The report consists of two sections – a graph and a table. On the graph there are blue bars representing the growth of the predicted PSSA performance group over the last tested year. The gold bars represent the growth of the three prior years’ groups of students in this predicted PSSA performance level.

District 8th grade Math This report provides evidence of how effective is the District at making academic progress with groups of students at different predicted achievement/performance levels. This is the grade level Performance Diagnostic Report. This report can be generated at a district or school level. The purpose of this report is to provide insight into the growth of students who are predicted to perform at the Below Basic, Basic, Proficient or Advanced levels on the current year’s exam in this subject. This prediction is based on the students’ previous performances. The report consists of two sections – a graph and a table. On the graph there are blue bars representing the growth of the predicted PSSA performance group over the last tested year. The gold bars represent the growth of the three prior years’ groups of students in this predicted PSSA performance level.

District 8th grade Reading This report provides evidence of how effective is the District at making academic progress with groups of students at different predicted achievement/performance levels. This is the grade level Performance Diagnostic Report. This report can be generated at a district or school level. The purpose of this report is to provide insight into the growth of students who are predicted to perform at the Below Basic, Basic, Proficient or Advanced levels on the current year’s exam in this subject. This prediction is based on the students’ previous performances. The report consists of two sections – a graph and a table. On the graph there are blue bars representing the growth of the predicted PSSA performance group over the last tested year. The gold bars represent the growth of the three prior years’ groups of students in this predicted PSSA performance level.

Upper Darby School District Growth Data PVAAS Growth Data: Developed and maintained by SAS Institute, Inc. Visit link below to access Pennsylvania School District data https://pvaas.sas.com/ District Data and School Data: Click to visit Public Site District Search, Select Upper Darby SD Individual School Data: Select School search, then select your school In this high-stakes assessment and accountability climate, we need accurate information to guide us in making appropriate decisions for students…decisions about instruction, curriculum, and assessment. PVAAS is one tool to assist schools in this process. PVAAS is an analysis of existing assessment data (PSSA) – no other testing is necessary. PVAAS utilizes longitudinal PSSA test scores to produce reliable estimates of growth. PVAAS is a tool to provide a measure of a school’s effectiveness from year to year. It provides progress data to add to achievement data as schools are analyzing their educational programs for students. PVAAS offers powerful diagnostic information about school practice by leveraging student achievement data with complex statistical tools. It provides a tool to identify strengths and weaknesses in educational programs.